search

cowinapi / developer.cowin

This group is created to facilitate technical and integration discussions related to cowin platform. API related contents can be obtained at API setu portal https://apisetu.gov.in/public/marketplace/api/cowin
115 stars 30 forks source link

Instead of Polling on the PublicAPI introduce a public Websocket endpoint which will publish the data only when the slots are available. #174

Open abhijeetsingh26 opened 3 years ago

abhijeetsingh26 commented 3 years ago

It will solve the following things

  1. Will reduce the un-necessary load on the schedules public API which is being polled by literally everyone who has the basic knowledge of hitting a endpoint and fetching/parsing the data. System is unnecessarily overburdened by these calls.

  2. No need of putting stringent rate limits on APIs as well as serving cached data saving the efforts.

  3. All the clients will have a real time view of the availability of the slots which will help reduce the gray area that people have around having some backdoors to the system for booking.

loflet commented 3 years ago

That is an interesting suggestion, just wondering from technical implementation - having so many centres is it possible to implement pub-sub model on this? If it is then definitely this should be a better approach.

abhijeetsingh26 commented 3 years ago

They can leave their internal implementation of how they get the slots updated as it is also.

They just need to run a scheduler which will internally poll their private endpoint and see if there has been a change.

While this isn't a fully pub-sub model from ground up but still since their internal scheduler is the only one that is polling their private endpoint, they can keep the polling frequency sufficiently low and keep the resource utilization minimal vs the current model.

Just from a quick google search on the max limits of concurrent websockets an application can handle I can say it shouldn't be a blocker to them.

https://alexhultman.medium.com/millions-of-active-websockets-with-node-js-7dc575746a01 https://blog.jayway.com/2015/04/13/600k-concurrent-websocket-connections-on-aws-using-node-js/

rudrasura commented 3 years ago

Instead of websockets which is bulky and has 2 way communication, why not to user Sever Sent Events? It is a very lightweight as compared to Web Sockets and 2 way communication for notification use case doesn't fit. A publisher subscriber model with Sever sent events is a good approach.

pulse-aj commented 3 years ago

Webhooks is a better option.

abhijeetsingh26 commented 3 years ago

IMHO webhooks will put down the service into same problems as it is having now i.e. "SCALING". As soon as they have sufficiently big number of webhooks to process, some will always trigger first.

Pub-Sub should be the way out so all clients received the data at the same time and if something at their end fails, all clients will have what we call as 'common-fate'.