In today’s fast-paced digital world, delivering news in real-time has become a fundamental expectation for users. This is especially true in the context of sports apps, where up-to-the-minute updates can make or break the user experience. But what does it take to build and maintain a system capable of delivering seamless news updates, push notifications, and personalized experiences to both iPhone and Android apps?<\/p>\n\n\n
This blog takes you behind the scenes of a recent project where we tackled this challenge head-on. From implementing AI-powered tagging to managing a distributed API capable of handling 100K+ requests per hour, here’s how we approached the complexities of delivering real-time news at scale.<\/p>\n\n\n\n
Before news even reaches the database, the editorial process ensures that every article is enriched with metadata and tags. To achieve this, we developed a NER (Named Entity Recognition) API<\/strong> that scans articles in real-time as editors work on them. The API identifies:<\/p>\n\n\n With a simple click, editors can select accurate tags, often better than they might choose manually. This system not only accelerates the tagging process but also ensures uniformity. Additionally, tags can automatically link players, teams, or competitions to detailed pages in the apps, creating a richer user experience.<\/p>\n\n\n\n Once an article is ready, it enters the Galera database cluster<\/strong>. Galera’s high availability ensures that articles are redundantly stored and immediately accessible. A cluster of web servers behind load balancers retrieves these articles and displays them on the website.<\/p>\n\n\n However, the story doesn’t stop at the website. The same information must also be prepared for distribution to iPhone and Android apps, requiring a separate, high-performance API.<\/p>\n\n\n\n Developing an API to handle news delivery and push notifications comes with its own set of challenges:<\/p>\n\n\n Building a scalable API isn’t just about coding—it’s about anticipating and mitigating issues during live operations. This is where monitoring played a crucial role:<\/p>\n\n\n With the API in place, the next challenge was delivering push notifications. We integrated Firebase Cloud Messaging (FCM)<\/strong> to manage push subscriptions for teams and ensure notifications reached users instantly. This system:<\/p>\n\n\n Deploying such a complex system always comes with risks. Despite the potential for errors, the system went live with only one minor hiccup—a few images failed to load initially. Other than that, the launch was flawless. The system now seamlessly delivers real-time news to apps, handles push notifications efficiently, and scales effortlessly to meet growing demand.<\/p>\n\n\n\n This project was a prime example of how back-end infrastructure, APIs, and front-end interfaces must work in harmony to deliver exceptional user experiences. Moving forward, we plan to:<\/p>\n\n\n Delivering real-time news to apps might seem simple to the end user, but as this story shows, it’s a symphony of carefully orchestrated systems and processes.<\/p>\n\n\n <\/p>\n","protected":false},"excerpt":{"rendered":" In today’s fast-paced digital world, delivering news in real-time has become a fundamental expectation for users. This is especially true in the context of sports apps, where up-to-the-minute updates can make or break the user experience. But what does it take to build and maintain a system capable of delivering seamless news updates, push notifications, …<\/p>\n","protected":false},"author":1,"featured_media":473,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[43,16],"tags":[25,17,37,27],"class_list":["post-472","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-chit-chat","category-tech-innovation","tag-high-available","tag-machine-learning","tag-monitoring","tag-redundancy"],"yoast_head":"\n\n
\n\n\nStep 2: From Publication to Website<\/strong><\/h4>\n\n\n
\n\n\nStep 3: Building a High-Performance API<\/strong><\/h4>\n\n\n
\n
With apps supporting push notifications for many teams and many thousands users consuming news in real-time, the API had to handle 100K+ requests per hour<\/strong>.<\/li>\n\n\n\n
Every SQL query needed careful thought. A single inefficient query could multiply into millions of extra requests, significantly impacting performance and costs.<\/li>\n\n\n\n
To balance speed and efficiency, we implemented a Redis cache<\/strong>, where each web server is part of a Redis cluster. This ensures scalability, redundancy, and local access to cached data. Cache times are fine-tuned based on the freshness requirements of different data types.<\/li>\n<\/ol>\n\n\n\n
\n\n\nStep 4: Monitoring and Fine-Tuning<\/strong><\/h4>\n\n\n
\n
Using tools like Grafana and our monitoring system, we tracked critical metrics such as 404<\/code> and 503<\/code> HTTP error rates and monitored for full message queues.<\/li>\n\n\n\n
Our monitoring system counted records in queues every minute, providing insights into potential bottlenecks before they became critical.<\/li>\n\n\n\n
Extensive testing ensured the system could handle peak traffic without failures. This included tuning Galera’s optimistic locking to minimize deadlocks under high concurrency and experimenting with techniques like splitting counters into 100 buckets to avoid bottlenecks.<\/li>\n<\/ul>\n\n\n\n
\n\n\nStep 5: Real-Time Push Notifications<\/strong><\/h4>\n\n\n
\n
\n\n\nStep 6: Deployment and Results<\/strong><\/h4>\n\n\n
\n\n\nLessons Learned<\/strong><\/h4>\n\n\n
\n
Building an API for high traffic requires meticulous query design, caching strategies, and concurrency handling.<\/li>\n\n\n\n
Real-time monitoring and visualization through Grafana provided critical insights during deployment and beyond.<\/li>\n\n\n\n
Solutions like splitting counters into buckets or fine-tuning cache times only emerged after extensive testing and experimentation.<\/li>\n\n\n\n
The NER API demonstrated how AI tools can make editorial processes faster, more accurate, and more uniform.<\/li>\n<\/ol>\n\n\n\n
\n\n\nLooking Ahead<\/strong><\/h4>\n\n\n
\n