Performance is absolutely critical when building software systems. Performance becomes a key differentiator in SaaS platforms like Healthcare platforms that want a competitive advantage.

Google and Amazon have both released independent numbers that inform us users on the key advantages of performance. In Amazon’s case, a 1 second delay in loading a page could result in $1.6B loss in sales. In Google’s case a slowdown of 4/10th of a second could lose 8 million in searches. Yes, performance matters.

At HyperTrends, we specialize in building Healthcare SaaS platforms that are highly performant, HIPAA compliant, modern and modular so that you can support any type of a business use case. Performance isn’t an afterthought, rather a competitive advantage.

Key Takeaways

• Performance is extremely important for SaaS platforms who want to maintain a competitive advantage
• Perform cost-benefit analysis around performance when building software. Performance comes with a cost.
• It is normal to iterate a few times to achieve the desired performance. Very few people understand that performance optimization is a process.
• Trade-offs are normal when planning for performance. Sometimes, you may end up spending a lot more to achieve that final performance advantage. Do you really need it?
• Redis is one-hell-of-a-platform – use it as much as possible. Read more here.

The Ask

At HyperTrends we needed to design an Events Calendar for one of our healthcare clients. The said service would load up events for multiple patients in a calendar format so that the administrators could quickly plan their day/week/month. Patients themselves would also have an opportunity to view the events on their personal calendar within a different virtual caregiver platform.

The creation, management, and retrieval of these complex recurring calendar events present a significant challenge. We wanted to be at parity with other proven calendar systems like Microsoft Outlook and Google but with the added benefit of being able to view an entire cohort on a single calendar.

Imagine a scenario where the system must handle hundreds of organizations, each with thousands of active subscribers. A naive developer might want to rely on a simple database system to accomplish this. And therein lies the problem. Our friend “Performance” comes screaming at you as soon as you launch.

KAUSHAL KHOKHAR – SOLUTIONS ARCHITECT, HYPERTRENDS GLOBAL INC.

The Limitations of RDS for High-Demand Event Management

In building the calendar events system for our healthcare client, we initially relied on Amazon RDS as our primary data store.

While RDS offers robust data integrity, scalability, and sophisticated query capabilities through SQL, we encountered several limitations, particularly when scaling to handle high transaction volumes and complex querying requirements inherent in our system.

1. Latency Issues: As the number of subscribers and events grew into the thousands, the latency in querying and retrieving event data from RDS increased significantly. During peak hours, when multiple staff members simultaneously created and managed events, the response time slowed, leading to a suboptimal user experience.
2. Read and Write Overheads: The complexity of managing and retrieving recurrent, overlapping, and highly interactive events put a considerable load on our RDS instance. SQL queries, especially those involving joins across multiple tables (events, users, and organizational data), became increasingly complex and time-consuming to execute.
3. Scalability Concerns: While RDS can be scaled vertically, this approach often leads to increased costs. Moreover, scaling vertically has its limits; beyond a certain point, additional hardware will not translate into proportional performance gains.
4. Cost of High Availability and Durability: Ensuring high availability and durability in RDS involves additional replication and backup mechanisms that further complicate the architecture and increase the cost, particularly for a system requiring a high degree of read and write operations distributed across a broad geographic area.

Why Redis Became Our Go-To Solution

To address these challenges, we turned to Redis, a lightning-fast in-memory data structure store, renowned for its performance and ease of use in high-throughput environments.

Here’s how Redis complemented our existing RDS setup to create a more efficient system:

1. Decreased Latency: By caching frequently accessed data, such as upcoming calendar events and subscriber-specific details, Redis drastically reduced the latency that plagued our RDS-based system. Event data could be fetched in milliseconds, significantly enhancing the responsiveness of our application.
2. Offloading Read Operations: Redis excels in handling high-volume read operations. We used Redis to serve high-read queries, thereby reducing the load on our RDS instance. This separation of concerns allowed us to optimize both systems according to their strengths—Redis for speed and RDS for storage and complex transactions.
3. Simplified Data Handling with Data Structures: Redis offers various data structures like hashes, sets, and sorted sets, which we leveraged to manage complex datasets such as recurring events efficiently. For example, sorted sets allowed us to maintain events in an ordered manner, making it easy to retrieve all events within a specific range quickly.
4. Cost-Effective Scaling: Scaling Redis by adding more nodes in a cluster or increasing the number of replicas is straightforward and does not incur the significant costs associated with scaling an RDS system. Additionally, because Redis operates in-memory, it can handle a larger volume of queries with fewer resources compared to disk-based databases.
5. Improved System Resilience: Using Redis as a caching layer added an extra level of resilience to our architecture. Even in cases where RDS might experience delays or temporary outages, the cached data in Redis ensures that the system remains operational, providing continuous access to critical event information.

Data Storing and Hydration Strategy

This section will cover how data is initially stored in Redis and the strategy for loading data into the cache. And there are two operations associated with it.

1. Event Data Storing

Event Data Storing involves saving detailed information about each calendar event into Redis. This data includes event attributes such as start and end times, title, description, and any other relevant information.

Implementation Example:

• Key: Each event is stored using a unique key that follows the pattern events:<event_id>.
• Data Structure: Redis hashes are used to store the attributes of each event. This allows for the efficient retrieval and update of individual fields.

event_data = {
    'title': 'Project Meeting',
    'start': '2023-03-15T09:00:00Z',
    'end': '2023-03-15T10:00:00Z',
    'description': 'Discuss project milestones',
    'organisation_id': 'org123',
    'subscriber_ids': ['sub1', 'sub2']
}
redis.hmset(f"events:{event_id}", event_data)

2. Sorted Sets for Event Indexing

This strategy uses Redis sorted sets to index events by their start times. This indexing supports efficient time-range queries, allowing for quick retrieval of events occurring within a specific timeframe.

Implementation Example:

• Subscriber Events Key: organization:<organization_id>:subscriber:<subscriber_id>:events – Sorted sets that index events for each subscriber within an organization, allowing for personalized event feeds.
• Scores and Values: The event’s start time timestamp serves as the score, and the event ID is the value.

Example Script:

# Indexing an event for individual subscribers
redis.zadd(f"organization:{organisation_id}:subscriber:{subscriber_id}:events", {event_id: start_timestamp})

# Indexing an event for an organization
redis.zadd(f"organization:{organisation_id}:events", {event_id: start_timestamp})

Cache Maintenance and Refreshment

Below, I outline steps for dehydrating and rehydrating the Redis cache, followed by detailed API flows for create, update, and delete operations.

Dehydration and Rehydration Strategy

• Dehydration: Involves removing stale or unused data from the cache. This can happen automatically through Redis expiration policies or through manual invalidation when data is updated or deleted.
• Rehydration: Involves loading or reloading data into the cache from the database. This happens after a cache miss during a read operation or after the cache has been dehydrated.

API Flow for Create, Update, and Delete Operations

1. Create (Write-Through Caching)

• Client Request: A request to create a new event is sent to the API.
• API Controller:
  • Validates and processes the request.
  • Simultaneously writes the event data to both the database and Redis cache.
  • In Redis, the event details are stored in a hash, and the event ID is added to the relevant sorted sets for indexing.
• Response: The API returns a success response with the created event details.

2. Update (Write-Through Caching)

• Client Request: A request to update an existing event is sent to the API.
• API Controller:
  • Validates and finds the event in the database.
  • Updates the event in the database.
  • Immediately updates the event details in Redis to ensure cache consistency.
  • Adjusts sorted set indexes if the start time has changed.
• Response: The API returns a success response with the updated event details.

3. Delete

• Client Request: A request to delete an event is sent to the API.
• API Controller:
  • Deletes the event from the database.
  • Removes the event details from the Redis cache, including its presence in any sorted set indexes.
• Response: The API confirms the deletion.

4. Retrieve (Cache-Aside/Lazy Loading)

• Client Request: A request to retrieve events, potentially triggering a cache miss.
• API Controller:
  • Checks if the event data is available in Redis.
  • Cache Hit: Returns the event data from the cache.
  • Cache Miss:
    • Retrieves the event data from the database.
    • Stores the retrieved data in Redis for future requests.
    • Returns the event data to the client.
• Response: The API returns the requested event details.

Cost-Benefit Analysis

Integrating Redis with RDS significantly reduces operational costs. For instance, achieving 30,000 queries per second (QPS) with RDS alone requires four read replicas at $1740/month, whereas a combination of RDS and Redis costs just $780/month, slashing expenses by 55%.

Metrics	RDS primary instance only	RDS with one read replica	RDS with 4 read replicas	ElastiCache with 1 read replica
Avg resp time	200ms	80ms	80ms	1ms
Avg read QPS	8000 QPS	16000 QPS	30000 QPS	32,000 QPS
Pricing $/month	$348 /month	$696 / month	$1740 / month	$780 / month
Nodes used:	1 read/write primary db.r6g.xlarge	1 writer 1 readers = 2x db.r6g.xlarge	1 writer 4 readers = 5x db.r6g.xlarge	1x db.r6g.xlarge + 2x cache.m6g.xlarge

The scalability of Redis not only enhances performance but also offers substantial cost advantages. With a fully warmed-up cache, Redis can handle up to 250,000 QPS, offering savings of 87% compared to the same throughput using RDS alone, underlining the financial efficiency of high-throughput, read-heavy applications.

Metrics	1 RDS + 9 read replicas	1 ElastiCache + 1 EC read replica (on 1 RDS + 1 read replica)
Avg resp time	80ms	9ms
Avg QPS	250,000	250,000
Pricing	$7,840/ month	$784 (RDS) + $432 (EC)/ month
Nodes used:	10 x db.r6g.xlarge	2 x db.r6g.xlarge + 2 x cache.m6g.xlarge

Conclusion

Incorporating Redis into our healthcare event management system has proven to be a transformative decision. By leveraging Redis alongside RDS, we have not only significantly enhanced the system’s performance but also achieved remarkable cost efficiencies. Redis’ ability to handle high throughput with low latency has enabled real-time data interactions that are crucial for read intensive scenarios.

The operational benefits of Redis extend beyond performance enhancements to substantial cost savings. As illustrated, the integration of Redis can reduce infrastructure costs by over 55% compared to using RDS alone. For high-throughput scenarios, the savings are even more pronounced, reaching up to 87%.

This journey through Redis implementation has underscored a vital lesson: the right technology can dramatically impact the functionality and financial health of healthcare systems. By sharing our experiences and the tangible benefits observed, we hope to inspire other organizations to explore how Redis can be tailored to meet their specific needs.

We invite you to engage with us in the comments below, share your experiences, or ask questions about integrating Redis into your systems. Let’s explore together how innovative data management solutions can transform the healthcare industry.