cowinapi / developer.cowin

This group is created to facilitate technical and integration discussions related to cowin platform. API related contents can be obtained at API setu portal https://apisetu.gov.in/public/marketplace/api/cowin
115 stars 30 forks source link

Cowin Portal Website slot availability late than API #399

Open saurajain1005 opened 3 years ago

saurajain1005 commented 3 years ago

Hello,

I am facing strange problem.

"https://cdn-api.co-vin.in/api/v2/appointment/sessions/public/findByDistrict"

This API is Identifying & returning slots well & accurate.

But even when i am logged in CoWin website at same time, slots are shown as 0 (0 booked, 0 available).

After 2-3 minutes, all slots are full on Website.

Tried refresh as soon as API reported, but website didnt show anything.

Why Website slot availability late for me?

Note: I am running API & Website on same laptop (same IP), but took care of API rate-limitation.

Please suggest, how to book slot, even when notified by API properly.

thanks.

kiranfegade19 commented 3 years ago

Only Protected Production API gives actual realtime data which matches with data what we see after login to cowin like Telegram.

Other Public API and Sandbox api are cached one.

nonrealjunco commented 3 years ago

It is really unfortunate that they have not lived up to their word. Guideline document clearly states the following in Annexure 1 :

  1. Public API a. Fetch Inventory of vaccination slots API: To allow third-party Apps to access in realtime the continuously updated list of available vaccination slots in Co-WIN . This will enable these apps to provide various options to the users like “look for available slots for vaccination / vaccines under multiple search criteria”.

I don't understand the brains behind caching appointment availability data.

If only Private API gets the real-time slot data then what is the use of Appointment Public API ? Only give Certificate API. Buy the time cached data is given to API consumers and the users slots are already gone. There are cases where 1200 slots are gone in 2-3 seconds. Many times even 1 available slot that got cancelled by someone else is useful.

Getting everyone vaccinated as quickly as possible is the aim right ?

We can understand Protected API's, data misuse and API abuse and rate limits. But caching data for 5 mins and allowing 100 requests / 5 mins is useless. I am going to get the same data mostly. If I use multiple instances I will get around this somewhat.

There are two Public API options available on registration page. Those who get API Keys should at least be allowed real-time data. Otherwise you defeat the purpose of an available API or for that matter this Public API.

saurajain1005 commented 3 years ago

Hi Guys,

Problem is the reverse..

In this case, Public API is giving data real-time. I was able to detect from public APIs, that slots got availble. Main problem is getting this reflected on Cowin Website.

But, when using that notification , checking slot on Browser website, there is none available. After some time, Browser website, made all slots consumed.

This is something with the implementation of the Cowin website & the browser caching.

My Guess:

  1. Caching in browser based on session with a time-out. (time out are critical now, since we are not getting more than 40-50 seconds to book 1 slot).
  2. Refresh (or Search button) on Browser is applied a limit, after which it is not sending call to real-backend APIs. Making situation worse (maybe a try for optimization for reducing server load).

In any case, its getting hard to sync Cowin-Website & APIs structure.

Any light on this, is highly appreciated. This is pandemic situation, vaccination is the key. Lets get it to the people as quickly as possible.

vinayakmoh commented 3 years ago

Hi Guys,

Problem is the reverse..

In this case, Public API is giving data real-time. I was able to detect from public APIs, that slots got availble. Main problem is getting this reflected on Cowin Website.

But, when using that notification , checking slot on Browser website, there is none available. After some time, Browser website, made all slots consumed.

This is something with the implementation of the Cowin website & the browser caching.

My Guess:

  1. Caching in browser based on session with a time-out. (time out are critical now, since we are not getting more than 40-50 seconds to book 1 slot).
  2. Refresh (or Search button) on Browser is applied a limit, after which it is not sending call to real-backend APIs. Making situation worse (maybe a try for optimization for reducing server load).

In any case, its getting hard to sync Cowin-Website & APIs structure.

Any light on this, is highly appreciated. This is pandemic situation, vaccination is the key. Lets get it to the people as quickly as possible.

There is a delay in Cowin Website, I don't believe it is browser Cache. I have tried this with multiple VM's in Azure. Some times the delay is in seconds but some times it is even mins. Issue I have faced is similar:

  1. Telegram alerts via data received from API about open slots for a given date
  2. Checked on Cowin - there is no availability of slot (booked and open) for the date given by API
  3. After Few sec/min - Hospital shows up with slot booked

Data Example 1: Center Name: ASTER RV HOSPITAL - JP NAGAR Center PinCode: 560078 Center FeeType: Paid Vaccine Date: 07-06-2021 Vaccine Name: COVISHIELD AgeLimit: 18 Dose 1 Capacity: 549 DataReceived From API: 5th June 2021 15:59:46 This only came in Cowin Website with fully booked at 16:00:05

Data Example 2: Center Name: ST THERESAS HOSPITAL Center Address: DR RAJ KUMAR ROAD 1ST BLOCK RAJAJINAGAR BANGALORE Center PinCode: 560010 Center FeeType: Paid Vaccine Date: 08-06-2021 Vaccine Name: COVISHIELD AgeLimit: 18 Dose 1 Capacity: 50 DataReceived From API: 5th June 2021 18:05:00 This information did not show up in website until 18:09:00 But later appeared in Cowin Website with fully booked at 18:09:20

API Data I am receiving is every 5 seconds, the slots were not reducing every 5 seconds but was cleared all at one shot. This basically means there is a backend upload of data.

The real problem is, it is difficult to get the real slots and alerts are generated based on slots that are not in sync with Cowin and API. Hope a resolution is given at the earliest.

saurajain1005 commented 3 years ago

Thanks Vinay. Yah I too agree with your opinion...


API Data I am receiving is every 5 seconds, the slots were not reducing every 5 seconds but was cleared all at one shot. This basically means there is a backend upload of data.

This means that Cowin portal backend is collecting & then uploading it to show on front-end.

But, who are booking these slots, in the meanwhile, how are they getting to see the slot-availability.

I tried looking from other system (on another network) too. It shows slot available after 30-40 sec, but when we try to book it, nothing available.

I think certainly there is some design issue, which is making this API nearly unusable.

But how a common man can book even from portal, when the slot display is this late.

And more important question - how is anybody getting slot to book, who & how are they booking.

3minus1 commented 3 years ago

+1

saurajain1005 commented 3 years ago

Check this link

https://economictimes.indiatimes.com/news/international/world-news/meet-the-vaccine-appointment-bots/administration-opposed/slideshow/81224685.cms

Healthcare officials are not very impressed with the bots, arguing that these pieces of code could bulk book appointments and in turn sell these slots for money. While there has been no evidence of such activity, counties are shutting down websites after people from far away booked appointments via bots.

I am sad, that this will block even a single user to book genuinely & promote black-marketing of slots even.

Why we should use technology for evil reasons :(

This is pandemic - Is money so important?

I know finally that slot is being used & given to someone. But again, this will make divide in our society.

nonrealjunco commented 3 years ago

Hi Guys, Problem is the reverse.. In this case, Public API is giving data real-time. I was able to detect from public APIs, that slots got availble. Main problem is getting this reflected on Cowin Website. But, when using that notification , checking slot on Browser website, there is none available. After some time, Browser website, made all slots consumed. This is something with the implementation of the Cowin website & the browser caching. My Guess:

  1. Caching in browser based on session with a time-out. (time out are critical now, since we are not getting more than 40-50 seconds to book 1 slot).
  2. Refresh (or Search button) on Browser is applied a limit, after which it is not sending call to real-backend APIs. Making situation worse (maybe a try for optimization for reducing server load).

In any case, its getting hard to sync Cowin-Website & APIs structure. Any light on this, is highly appreciated. This is pandemic situation, vaccination is the key. Lets get it to the people as quickly as possible.

There is a delay in Cowin Website, I don't believe it is browser Cache. I have tried this with multiple VM's in Azure. Some times the delay is in seconds but some times it is even mins. Issue I have faced is similar:

  1. Telegram alerts via data received from API about open slots for a given date
  2. Checked on Cowin - there is no availability of slot (booked and open) for the date given by API
  3. After Few sec/min - Hospital shows up with slot booked

Data Example 1: Center Name: ASTER RV HOSPITAL - JP NAGAR Center PinCode: 560078 Center FeeType: Paid Vaccine Date: 07-06-2021 Vaccine Name: COVISHIELD AgeLimit: 18 Dose 1 Capacity: 549 DataReceived From API: 5th June 2021 15:59:46 This only came in Cowin Website with fully booked at 16:00:05

Data Example 2: Center Name: ST THERESAS HOSPITAL Center Address: DR RAJ KUMAR ROAD 1ST BLOCK RAJAJINAGAR BANGALORE Center PinCode: 560010 Center FeeType: Paid Vaccine Date: 08-06-2021 Vaccine Name: COVISHIELD AgeLimit: 18 Dose 1 Capacity: 50 DataReceived From API: 5th June 2021 18:05:00 This information did not show up in website until 18:09:00 But later appeared in Cowin Website with fully booked at 18:09:20

API Data I am receiving is every 5 seconds, the slots were not reducing every 5 seconds but was cleared all at one shot. This basically means there is a backend upload of data.

The real problem is, it is difficult to get the real slots and alerts are generated slots that are not in sync with Cowin and API. Hope a resolution is given at the earliest.

Sometimes all slots get cleared if the population density is high in the area or the hospital is well known. It has nothing to do with the api. When a telegram channel has 25000 users. A slot with 1000 doses can disappear in seconds.

And with an unknown hospital you will still get the slots with reduced number. I have seen it.

EG:

Channel with Private API gets data At 10:37 AM

Pincode: Name: Vaccine: COVISHIELD Date: 07-06-2021 Age: 18+ Dose 1: 95 Dose 2: 5

Channel with Public API gets data At 10:40 AM

Age : 18+ Hospital Name Pincode Date : 07-06-2021 COVISHIELD Dose 1 : 0 Dose 2 : 5

The only problem is live data should be given to all.

bertythomas commented 3 years ago

Guys, this problem is real. Telegram alerts are using public APIs only. They are giving accurate and up to date data. The problem is the cache on the CoWIN portal side. It sometimes takes minutes for the slots to reflect on the CoWIN portal after it first actually appears as seen via the public APIs. A prime example of this delay is Ernakulam district in Kerala. It takes minutes for the slots to actually show up on the CoWIN portal. However, I should also admit that this delay is not uniform. Some will get the request served with cache of about a few seconds and some with cache of minutes for the same district/ date. Can you check if there is a difference in cache times across your AWS instances? I don't know how else to explain that not everyone gets the same cache delay.

saurajain1005 commented 3 years ago

Yah Thomas, I agree with you.

This is happening quite frequent, and rarely we can see, slots available on API on real Cowin portal. This is making any solution nearly redundant.

I think atleast there must be a defined logic for cache invalidation. For a new session login, atleast portal should show correct & live data.

Dear Cowin portal developers, please take a note.

vsingal-p commented 3 years ago

@cowinapi This problem is hitting real hard. API shows availability for a centre which is not even visible on live portal. How can one book slots if it's not visible? Been facing this for past 3 hours on multiple devices and IPs

vsingal-p commented 3 years ago

Even the data shown on by pincode and by district are different on website for the same location and instant of time. How can one book slots if theirs so much inconsistency everywhere in the system?

saurajain1005 commented 3 years ago

Yes guys, faced all these observations.

1 more:

Sometimes, a device on a different IP, not logged from 30 minutes, get to show some slots when logged in freshly.

Indicating that there is a (session, ip) related key used in cache expiry logic.

Mitalee commented 3 years ago

Even the data shown on by pincode and by district are different on website for the same location and instant of time. How can one book slots if theirs so much inconsistency everywhere in the system?

This is exactly right. Searching by pin code does not show the slot availability. Have to go by state/district option.

pankajbatra commented 3 years ago

Even on a browser session, getting 403 after a while with following message {"message": "User is not authorized to access this resource with an explicit deny"}

pankajbatra commented 3 years ago

Yes guys, faced all these observations.

1 more:

Sometimes, a device on a different IP, not logged from 30 minutes, get to show some slots when logged in freshly.

Indicating that there is a (session, ip) related key used in cache expiry logic.

Yes, looks like cache is IP specific.

Deepanshuarora20 commented 3 years ago

Any solution as this problem is not even allowing to book slots which open 500 at a time and are available for a minute

Practically either we dont see the center/slot or see it booked

ankit21990 commented 3 years ago

Any idea what is happening here, few centers are not even visible on cowin portal but getting free slots for those locations from APIs. How slots are getting booked for those centers which are not even present on cowin portal

Deepanshuarora20 commented 3 years ago

@cowinapi any suggestions why this is happening and how we can correct this

saurajain1005 commented 3 years ago

Any idea what is happening here, few centers are not even visible on cowin portal but getting free slots for those locations from APIs. How slots are getting booked for those centers which are not even present on cowin portal

---- client side caching logic based on IP. By some random logic, sometimes it may get visible too. But this makes system unpredictable.