[GUIDE] How to Overcome the new API Changes to Refresh/IP Throttle after 10 tries

SRvSaha commented 3 years ago

[UPDATE] Issue Resolved in v5.2.0 Release

Don't Bombard CoWIN Servers with huge number of requests, if you are using the App, in that way you might be in trouble. In case you are using it, use with precaution. You may get idea of openings of slots from historical data available here: https://under45.in/reports.php or from some alerts/notifications like Telegram or by running this App (unless you are bombarding with 1/2 hours of searching for slots). Lately, found that this service http://vaccineslot.in/ provides very good data analytics of Slots in Across India in every district/state.

Reference Suggestion: https://github.com/bombardier-gif/covid-vaccine-booking/issues/489#issuecomment-857032762

Due to some new changes in the CoWIN APIs there has been the following observations:

Seems like they removed Captcha requirement for Booking
They have added some hard refresh limit of the APIs so that users can't keep on hitting their server. Currently, based on User Inputs and analysis, it has been found that the Limit is of 10/20 Requests after which IP is throttled. {'message': 'User is not authorized to access this resource with an explicit deny'}

So, a temporary workaround to go around this for the moment till we get a solution is this:

This is a two step Process: FindingSlot and BookingSlot.

Step For Finding Slot:

Run the Application as usual (CoWinVaccineSlotFinder.exe for Windows, or run.sh for Linux/Mac) and then after you receive the OTP and enter the OTP, you'll be shown that your OTP is stored in authToken.json file. Once that is done, close the application.

There is an option in the config file appsettings.json "ProtectedAPI": { "IsToBeUsed": true CHANGE it to "IsToBeUsed": false This would make use of the Public API to find the Slots for the meantime, since there is no hard limit of 10/20 requests. It'll work till 100 requests/5 minutes.
After making the change in the Config, save it and Run the Application (CoWinVaccineSlotFinder.exe for Windows, or run.sh for Linux/Mac) and let it run until it finds a slot. Proof below that it runs even after 10 tries as well.
After the Application starts running, again go to the config file and CHANGE it to "IsToBeUsed": true . We are doing this because for Booking of Slot we need the Protected API and we don't want to waste any time when slot is found
Following will be shown when you run the Application as it will keep on searching and when Slot is found it'll crash the application as booking is not allowed using Public API. You just need to close the Application and following the next step for booking.

Step For Booking Slot

As soon as you find slot is available from the above step, close the Running Application as the Application will automatically crash.
Make sure "ProtectedAPI": { "IsToBeUsed": true is set in the config file appsettings.json
Run the Application once again (CoWinVaccineSlotFinder.exe for Windows, or run.sh for Linux/Mac)
This will not need the OTP again as your OTP is already being asked in the above step and your authentication credentials are saved in authToken.json.
You'll right away see the slot and your booking will be done. Since Captcha is still not removed from Code, so use "AutoReadCaptcha": true

This is just a hot-fix without making any changes in the Code. Please do let us know if this trick works so that we can work on fixing it and releasing a new release version.

SRvSaha commented 3 years ago

@Suren1978 Have you tried by changing the config as suggested in the description. Please do that and inform. Otherwise it won't work directly due to changes in the API made by COWIN Team this evening.

Suren1978 commented 3 years ago

actually it could not establish session with public API

520rahul commented 3 years ago

I tried the above suggested method but facing a new problem as it could not establish session even i m entering correct and valid OTP (see above image)

@Suren1978 bro. Hide ur mobile no

Suren1978 commented 3 years ago

I tried the above suggested method but facing a new problem as it could not establish session even i m entering correct and valid OTP (see above image)

@Suren1978 bro. Hide ur mobile no

bro pls delete image in your reply

Suren1978 commented 3 years ago

bro pls delete images i send ...it is request

On Thu, Jun 3, 2021, 23:12 520rahul @.***> wrote:

[image: Screenshot (21)] https://user-images.githubusercontent.com/85307388/120686047-31025980-c4be-11eb-9edd-57e3ead4b328.png I tried the above suggested method but facing a new problem as it could not establish session even i m entering correct and valid OTP (see above image)

@Suren1978 https://github.com/Suren1978 bro. Hide ur mobile no

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/SRvSaha/CoWinVaccineSlotFinder/issues/79#issuecomment-854057981, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUK277CIOECOYGPMDYLF7N3TQ65HHANCNFSM46BFFU2Q .

Tnu02 commented 3 years ago

Not sure but in the evening when I tried using the 5.1 for Autocaptcha set as true, it entered captcha automatically and didnt book the slot (as the captcha was disabled on the cowin by default) but again i tried setting the autocaptcha as false, then it popped up captcha and when i entered it, slot got booked.

is it somehow related @SRvSaha

govindasomani commented 3 years ago

This will have to be a 3 step process I guess...

Let the Protected API be set to true for login
After login, close the app. Change it to false and then open again.
When slot opens, close the app, change the protected API to true and open again.

I tried doing this, and not facing any login related issues.

Suren1978 commented 3 years ago

This will have to be a 3 step process I guess...

Let the Protected API be set to true for login

After login, close the app. Change it to false and then open again.

When slot opens, close the app, change the protected API to true and open again.

I tried doing this, and not facing any login related issues.

Correct it is working in this way only but 10 tries are fixed even you switch between true nfalse

520rahul commented 3 years ago

bro pls delete images i send ...it is request …

Sorry. I couldn’t see it. I have deleted the same

Suren1978 commented 3 years ago

Every time you receive a new OTP ...Cowin will give you 10 tries only ...therefore authToken file doesn't work ....you just remove this file to generate OTP again and then only Cowin portal give you another 10 chance to hit.....is there any solution of this problem....

SRvSaha commented 3 years ago

Not sure but in the evening when I tried using the 5.1 for Autocaptcha set as true, it entered captcha automatically and didnt book the slot (as the captcha was disabled on the cowin by default) but again i tried setting the autocaptcha as false, then it popped up captcha and when i entered it, slot got booked.

is it somehow related @SRvSaha

No. Seems like captcha is disabled by CoWin. If possible share screnshot from CoWIN Portal so that Captcha can be removed from Code. Even through they removed captcha, there is no issue with Auto-Captcha, booking still work. Only issue is due to very harsh limits for Protected APIs to stop API-Abuse.

SRvSaha commented 3 years ago

This will have to be a 3 step process I guess...
1. Let the Protected API be set to true for login

2. After login, close the app. Change it to false and then open again.

3. When slot opens, close the app, change the protected API to true and open again.
I tried doing this, and not facing any login related issues.

Perfect! This is correct. Actually this is what I was trying to explain in detail. @govindasomani @520rahul @Suren1978 @Tnu02

Tnu02 commented 3 years ago

This will have to be a 3 step process I guess...
1. Let the Protected API be set to true for login

2. After login, close the app. Change it to false and then open again.

3. When slot opens, close the app, change the protected API to true and open again.
I tried doing this, and not facing any login related issues.
Perfect! This is correct. Actually this is what I was trying to explain in detail. @govindasomani @520rahul @Suren1978 @Tnu02

Let me try this...

SRvSaha commented 3 years ago

Every time you receive a new OTP ...Cowin will give you 10 tries only ...therefore authToken file doesn't work ....you just remove this file to generate OTP again and then only Cowin portal give you another 10 chance to hit.....is there any solution of this problem....

See, AuthToken is needed only for Protected APIs. We are trying to shift the searching of slots part from Protected APIs (which needs AuthToken) to Public APIs (which doesn't need Auth Token, but are rate limited to 100/5 mins). In this way, we will have our Session Logged In using the AuthToken, so that when we get a slot using the Public API, we close the App, change the setting to use the Protected API and then booking can get done without waiting for OTP

govindasomani commented 3 years ago

Every time you receive a new OTP ...Cowin will give you 10 tries only ...therefore authToken file doesn't work ....you just remove this file to generate OTP again and then only Cowin portal give you another 10 chance to hit.....is there any solution of this problem....

See, AuthToken is needed only for Protected APIs. We are trying to shift the searching of slots part from Protected APIs (which needs AuthToken) to Public APIs (which doesn't need Auth Token, but are rate limited to 100/5 mins). In this way, we will have our Session Logged In using the AuthToken, so that when we get a slot using the Public API, we close the App, change the setting to use the Protected API and then booking can get done without waiting for OTP

The only problem that can come with this, is that because public API is cached, you may not get the slot immediately after it opens. And if there is too much time gap between the slot opening and showing on public API, then those might get booked too. As a workaround, you can shift from public to protected API once you see a telegram notification of slot opening. Just a thought, have not tested this.

SRvSaha commented 3 years ago

Every time you receive a new OTP ...Cowin will give you 10 tries only ...therefore authToken file doesn't work ....you just remove this file to generate OTP again and then only Cowin portal give you another 10 chance to hit.....is there any solution of this problem....

See, AuthToken is needed only for Protected APIs. We are trying to shift the searching of slots part from Protected APIs (which needs AuthToken) to Public APIs (which doesn't need Auth Token, but are rate limited to 100/5 mins). In this way, we will have our Session Logged In using the AuthToken, so that when we get a slot using the Public API, we close the App, change the setting to use the Protected API and then booking can get done without waiting for OTP

The only problem that can come with this, is that because public API is cached, you may not get the slot immediately after it opens. And if there is too much time gap between the slot opening and showing on public API, then those might get booked too. As a workaround, you can shift from public to protected API once you see a telegram notification of slot opening. Just a thought, have not tested this.

True. But usually Public APIs also get more or less data at similar times as 99% of Telegram or other notifiers using Public APIs only for notifying.

SRvSaha commented 3 years ago

This will have to be a 3 step process I guess...
1. Let the Protected API be set to true for login

2. After login, close the app. Change it to false and then open again.

3. When slot opens, close the app, change the protected API to true and open again.
I tried doing this, and not facing any login related issues.
Perfect! This is correct. Actually this is what I was trying to explain in detail. @govindasomani @520rahul @Suren1978 @Tnu02

I am updating the description with screenshots

Tnu02 commented 3 years ago

Getting catpcha while booking with 5.1 and not on manual.

SRvSaha commented 3 years ago

@Tnu02 @520rahul @Suren1978 Screenshots updated and tested in personal system, found it to be working. Try it out as per the screenshots and instructions in the description of the Issue. @govindasomani Thanks for pointing out

SRvSaha commented 3 years ago

Getting catpcha while booking with 5.1 and not on manual.

Make sure you have set this properly the setting "AutoReadCaptcha": true

Suren1978 commented 3 years ago

@Tnu02 @520rahul @Suren1978 Screenshots updated and tested in personal system, found it to be working. Try it out as per the screenshots and instructions in the description of the Issue. @govindasomani Thanks for pointing out

I hv already tried and mentioned to @govindasomani that this is working in this way only...

SRvSaha commented 3 years ago

@Tnu02 Cool. Had to delete the comments as it had the mobile number. See the updated Description. @Suren1978 Yeah, updated the steps mentioned by @govindasomani in the description.

Suren1978 commented 3 years ago

Screenshot (24)

I hv just tried for booking a slot for 45+ on trial basis ...no captcha was needed for this....I am not sure that it was previously needed for 45+ before amending in API

SRvSaha commented 3 years ago

I hv just tried for booking a slot for 45+ on trial basis ...no captcha was needed for this....I am not sure that it was previously needed for 45+ before amending in API

Okay cool @Suren1978, so now it is confirm that they removed Captcha. Can you please go to the Network Tab and then Select XHR like below, that will make sure what is being passed in the API. It was needed Previously. They made changes today evening

SRvSaha commented 3 years ago

Getting catpcha while booking with 5.1 and not on manual.

Make sure you have set this properly the setting "AutoReadCaptcha": true

Sorry I had to delete your comment.

Just curious, could you book the slot after following the steps?

Tnu02 commented 3 years ago

I hv just tried for booking a slot for 45+ on trial basis ...no captcha was needed for this....I am not sure that it was previously needed for 45+ before amending in API

Okay cool @Suren1978, so now it is confirm that they removed Captcha. Can you please go to the Network Tab and then Select XHR like below, that will make sure what is being passed in the API. It was needed Previously. They made changes today evening

They removed captcha for manual process but still we get it when booking thru the exe

Tnu02 commented 3 years ago

Getting catpcha while booking with 5.1 and not on manual.

Make sure you have set this properly the setting "AutoReadCaptcha": true

Sorry I had to delete your comment.

Just curious, could you book the slot after following the steps?

Yes.. Checked with both manually entering captcha and autocaptcha true...

SRvSaha commented 3 years ago

@Tnu02 @520rahul @Suren1978 @govindasomani Guys need your feedback on the following:

Is the current description working? We need to test it and then I will make changes and release v5.1.1 where these things will be taken care automatically.
What I am planning to do is: By Default after OTP verification all searching will be based on Public APIs and then Booking will be tried using Protected APIs. What say? Do you see any flaws in this?
We will remove the captcha from the Application so it works without it and saves some time.

Tnu02 commented 3 years ago

@Tnu02 @520rahul @Suren1978 @govindasomani Guys need your feedback on the following:

Is the current description working? We need to test it and then I will make changes and release v5.1.1 where these things will be taken case automatically.

What I am planning to do is: By Default after OTP verification all searching will be based on Public APIs and then Booking will be tried using Protected APIs. What say? Do you see any flaws in this?

We will remove the captcha from the Application so it works without it and saves some time.

@SRvSaha even tho autocaptcha is set to false, still getting the pop up for it..

SRvSaha commented 3 years ago

I hv just tried for booking a slot for 45+ on trial basis ...no captcha was needed for this....I am not sure that it was previously needed for 45+ before amending in API

Okay cool @Suren1978, so now it is confirm that they removed Captcha. Can you please go to the Network Tab and then Select XHR like below, that will make sure what is being passed in the API. It was needed Previously. They made changes today evening

They removed captcha for manual process but still we get it when booking thru the exe

Yes it is because we have not removed it yet from the Application yet. Who knows if they again come up with Captcha tomorrow? Also, removing captcha is not tough, just need assurance that it works with Captcha as well that that users can still book without upgrade till a new version will all the fixes are done.

govindasomani commented 3 years ago

@Tnu02 @520rahul @Suren1978 @govindasomani Guys need your feedback on the following:

Is the current description working? We need to test it and then I will make changes and release v5.1.1 where these things will be taken case automatically.

What I am planning to do is: By Default after OTP verification all searching will be based on Public APIs and then Booking will be tried using Protected APIs

The description is perfect. Need to test it in a live environment when slots are releasing. Conceptually though, there seems to be no problem with the flow.

Tnu02 commented 3 years ago

@Tnu02 @520rahul @Suren1978 @govindasomani Guys need your feedback on the following:

Is the current description working? We need to test it and then I will make changes and release v5.1.1 where these things will be taken case automatically.

What I am planning to do is: By Default after OTP verification all searching will be based on Public APIs and then Booking will be tried using Protected APIs

The description is perfect. Need to test it in a live environment when slots are releasing. Conceptually though, there seems to be no problem with the flow.

Agree with @govindasomani

SRvSaha commented 3 years ago

@Tnu02 @520rahul @Suren1978 @govindasomani Guys need your feedback on the following:

Is the current description working? We need to test it and then I will make changes and release v5.1.1 where these things will be taken case automatically.

What I am planning to do is: By Default after OTP verification all searching will be based on Public APIs and then Booking will be tried using Protected APIs. What say? Do you see any flaws in this?

We will remove the captcha from the Application so it works without it and saves some time.

@SRvSaha even tho autocaptcha is set to false, still getting the pop up for it..

@Tnu02 Yes you are supposed to get. False => Manual Captcha, true => Auto Captcha

Tnu02 commented 3 years ago

@Tnu02 @520rahul @Suren1978 @govindasomani Guys need your feedback on the following:

Is the current description working? We need to test it and then I will make changes and release v5.1.1 where these things will be taken case automatically.

What I am planning to do is: By Default after OTP verification all searching will be based on Public APIs and then Booking will be tried using Protected APIs. What say? Do you see any flaws in this?

We will remove the captcha from the Application so it works without it and saves some time.

@SRvSaha even tho autocaptcha is set to false, still getting the pop up for it..

@Tnu02 Yes you are supposed to get. False => Manual Captcha, true => Auto Captcha

Thou they removed it for manual user !!

SRvSaha commented 3 years ago

@Tnu02 @520rahul @Suren1978 @govindasomani Guys need your feedback on the following:

Is the current description working? We need to test it and then I will make changes and release v5.1.1 where these things will be taken case automatically.

What I am planning to do is: By Default after OTP verification all searching will be based on Public APIs and then Booking will be tried using Protected APIs

The description is perfect. Need to test it in a live environment when slots are releasing. Conceptually though, there seems to be no problem with the flow.

Agree with @govindasomani

@govindasomani @Tnu02 @520rahul @Suren1978 Okay cool. Let's wait till tomorrow 8 pm, since most bookings come from 4pm to 8 pm. See if you can do the testing and tally the results will Telegram/other realtime notifiers if we are getting the slots in correct time. And then is booking successful. If these things are okay, I will make the changes and release by EOD around 11 pm when traffic is low. Your support and comment please?

520rahul commented 3 years ago

@SRvSaha i think this will work. I shall tell u my experience. Thanks again brother. And eagerly waiting for v 5.1.1 as mentioned by u. God bless u

SRvSaha commented 3 years ago

3\. We will remove the captcha from the Application so it works without it and saves some time.

Read this part: "We will remove the captcha from the Application so it works without it and saves some time." They have removed it (hidden it). But since our Code still has it so you are getting the Captcha. That's why I am saying it is not hampering the core functionality though, as bookings are still successful. So, we can remove it only after testing of it as mentioned in previous chat

govindasomani commented 3 years ago

@Tnu02 @520rahul @Suren1978 @govindasomani Guys need your feedback on the following:

Is the current description working? We need to test it and then I will make changes and release v5.1.1 where these things will be taken case automatically.

What I am planning to do is: By Default after OTP verification all searching will be based on Public APIs and then Booking will be tried using Protected APIs

The description is perfect. Need to test it in a live environment when slots are releasing. Conceptually though, there seems to be no problem with the flow.

Agree with @govindasomani

@govindasomani @Tnu02 @520rahul @Suren1978 Okay cool. Let's wait till tomorrow 8 pm, since most bookings come from 4pm to 8 pm. See if you can do the testing and tally the results will Telegram/other realtime notifiers if we are getting the slots in correct time. And then is booking successful. If these things are okay, I will make the changes and release by EOD around 11 pm when traffic is low. Your support and comment please?

Sure. Will keep you posted on the experience.

SRvSaha commented 3 years ago

@SRvSaha i think this will work. I shall tell u my experience. Thanks again brother. And eagerly waiting for v 5.1.1 as mentioned by u. God bless u

Thanks @520rahul. Please do testing in the live environment along with all details. I don't want the release to hamper bookings of users.

Tnu02 commented 3 years ago

@Tnu02 @520rahul @Suren1978 @govindasomani Guys need your feedback on the following:

Is the current description working? We need to test it and then I will make changes and release v5.1.1 where these things will be taken case automatically.

What I am planning to do is: By Default after OTP verification all searching will be based on Public APIs and then Booking will be tried using Protected APIs

The description is perfect. Need to test it in a live environment when slots are releasing. Conceptually though, there seems to be no problem with the flow.

Agree with @govindasomani

@govindasomani @Tnu02 @520rahul @Suren1978 Okay cool. Let's wait till tomorrow 8 pm, since most bookings come from 4pm to 8 pm. See if you can do the testing and tally the results will Telegram/other realtime notifiers if we are getting the slots in correct time. And then is booking successful. If these things are okay, I will make the changes and release by EOD around 11 pm when traffic is low. Your support and comment please?

As of now it seems that it is working but ya need to check in real time scenario and then only would able to share with you @SRvSaha . Thanks

SRvSaha commented 3 years ago

@Tnu02 @520rahul @Suren1978 @govindasomani Guys need your feedback on the following:

Is the current description working? We need to test it and then I will make changes and release v5.1.1 where these things will be taken case automatically.

What I am planning to do is: By Default after OTP verification all searching will be based on Public APIs and then Booking will be tried using Protected APIs

The description is perfect. Need to test it in a live environment when slots are releasing. Conceptually though, there seems to be no problem with the flow.

Agree with @govindasomani

@govindasomani @Tnu02 @520rahul @Suren1978 Okay cool. Let's wait till tomorrow 8 pm, since most bookings come from 4pm to 8 pm. See if you can do the testing and tally the results will Telegram/other realtime notifiers if we are getting the slots in correct time. And then is booking successful. If these things are okay, I will make the changes and release by EOD around 11 pm when traffic is low. Your support and comment please?

As of now it seems that it is working but ya need to check in real time scenario and then only would able to share with you @SRvSaha . Thanks

Yes do test and inform by 20:00 Hrs tomorrow.

Suren1978 commented 3 years ago

Screenshot (26)

the above is the image without asking for captcha but I hv not confirmed the booking

SRvSaha commented 3 years ago

the above is the image without asking for captcha but I hv not confirmed the booking

Ok cool. Seems like they aren't calling the captcha endpoint. Do the testing of the application in live booking and let me know. We'll make the fixes.

520rahul commented 3 years ago

the above is the image without asking for captcha but I hv not confirmed the booking

Ok cool. Seems like they aren't calling the captcha endpoint. Do the testing of the application in live booking and let me know. We'll make the fixes.

Sure boss. @SRvSaha i will keep u posted after checking in real time situation so that u can modify the new version accordingly

SRvSaha commented 3 years ago

@govindasomani @Tnu02 @520rahul @Suren1978 At the time of Testing, please change the URLs to use findByPIN and findByDistrict as well. Based on user feedback from multiple places, people are telling that findByDistrict usually is used in Telegram as it has near real-time data and doesn't cache.

I am adding the block here: "FetchCalenderByDistrictUrl": "https://cdn-api.co-vin.in/api/v2/appointment/sessions/public/findByDistrict", "FetchCalenderByPINUrl": "https://cdn-api.co-vin.in/api/v2/appointment/sessions/public/findByPin"

Tnu02 commented 3 years ago

Just trying again with autocaptcha set as false.. pop up appear for captcha and if we cancel it then also it is booking the slot...

SRvSaha commented 3 years ago

Just trying again with autocaptcha set as false.. pop up appear for captcha and if we cancel it then also it is booking the slot...

Yeah @Tnu02, that is happening because when you are closing the Captcha Popup, Captcha Value is Blank. But since the API doesn't consider the Captcha Value for Booking, hence it works. Same is the reason when Auto-Captcha is turned on. It computes and sends the captcha, however it is neglected at the Server Side that's why our Application is still working. Otherwise it would have broken.

520rahul commented 3 years ago

@SRvSaha boss i m facing 2 issues-

When i changed public api to find by pin and find by district then error shown as per screenshot attahced.
When i changed it back to calendar by pin and district then though it was searching for slots but even i set my refresh limit to 3000 millisecods. It started giving me fatel error even on public api and even after restarting the application fatel errror did not stopped

SRvSaha commented 3 years ago

@govindasomani @Tnu02 @520rahul @Suren1978 At the time of Testing, please change the URLs to use findByPIN and findByDistrict as well. Based on user feedback from multiple places, people are telling that findByDistrict usually is used in Telegram as it has near real-time data and doesn't cache.

I am adding the block here: "FetchCalenderByDistrictUrl": "https://cdn-api.co-vin.in/api/v2/appointment/sessions/public/findByDistrict", "FetchCalenderByPINUrl": "https://cdn-api.co-vin.in/api/v2/appointment/sessions/public/findByPin"

Guys, Saw discussions that only CalenderAPIs has the hard refresh limit of 10/20. For FindAPIs it is not there.

These needs to be test from my side only as the Application will throw error while trying to call FindAPIs due to difference in the response structure of the API.

Test Case 1:

    "ProtectedAPI": {
      "IsToBeUsed": true,
      "FetchCalenderByDistrictUrl": "https://cdn-api.co-vin.in/api/v2/appointment/sessions/findByDistrict",
      "FetchCalenderByPINUrl": "https://cdn-api.co-vin.in/api/v2/appointment/sessions/findByPin"
    }

Test Case 2:

    "PublicAPI": {
      "FetchCalenderByDistrictUrl": "https://cdn-api.co-vin.in/api/v2/appointment/sessions/public/findByDistrict",
      "FetchCalenderByPINUrl": "https://cdn-api.co-vin.in/api/v2/appointment/sessions/public/findByPin"
    },
   "ProtectedAPI": {
      "IsToBeUsed": false,
   }

https://cdn-api.co-vin.in/api/v2/appointment/sessions/findByDistrict

520rahul commented 3 years ago

5ACADF6D-9E7C-4E70-9216-EFFB666B3F27

this is the error as mentioned in point 1

SRvSaha / CoWinVaccineSlotFinder