Blog

piStats Lambda Architecture

13/Oct/2017 Posted By admin no comments.
Tags ,

Problem Statement

piStats enables it’s clients to analyse the users on their site and send Push Notifications to engage and retain them for a longer duration. For this purpose the clients’ register their users’ on piStats using piStats SDKs.

The user base registered on piStats is unevenly distributed across different clients and their properties and the registration load is unpredictable – it is usually high for either first release with piStats integration or on a day when a special event occurs like elections.

To handle the unpredictable load piStats needs to scale up or down as the need occurs.

Solution and Flow

To handle the load of sudden spurt of registration and subscription and to remove the overhead of scaling up and down to hand in accordance with the load AWS Lambda is being utilised by piStats.

Whenever a new client is on boarded with piStats and a release is rolled out by that client after integration, piStats observes a sudden surge of events. With this surge of event AWS Lambda scales up easily and handles all events without us having to intervene and removes the overhead of handling the load manually.

When the load reduces Lambda scales down itself and reduces the number of servers that were started to handle the sudden spurt of events.

Components:

  • Kong
  • AWS Lambda
  • Redis
  • AWS RDS
  • Kinesis

Flow

Any request hitting piStats landa onto the nginx servers with ELB in front with autoscaling to handle unpredictable load of events. The nginx servers then forward the registration/subscription requests to Lambda where all the interaction with Redis, RDS and FCM(Firebase Cloud Messaging) occurs.

After processing these requests are again sent to the nginx servers which in turn send thses requests to Kinesis for further processing by our storm servers deployed on EC2 machines.
The storm servers perform the required enrichment and send the data for storage in our analytics database.

Anjna Bhati, Software Engineer

Anjna Bhati is Software Engineer with BluePi Consulting with expertise in cutting-edge technology, and has an extensive skill-set. She is currently working on feature enhancement of piStats, a super-hero suite of products for media and entertainment houses.

Category : AWS Big Data Lambda
Newsletter