search

opentargets / issues

Issue tracker for Open Targets Platform and Open Targets Genetics Portal
https://platform.opentargets.org https://genetics.opentargets.org
Apache License 2.0
12 stars 2 forks source link

Public portal down? #402

Closed pieterlukasse closed 5 years ago

pieterlukasse commented 5 years ago

See screenshot below:

image

Error details:

description: "Too many requests. Please look at the "X-Usage-Limit-Wait" header for the time to wait for an other call"
pieterlukasse commented 5 years ago

seems to be an issue on the public portal only. Is working on my local installation.

pieterlukasse commented 5 years ago

hm...portal seems to be down. This is how profile page looks now

image

deniseOme commented 5 years ago

Thanks @pieterlukasse. I can confirm the behaviour (spinning wheel does not stop spinning). screen shot 2019-01-15 at 13 51 18.

Will look into it...

andrewhercules commented 5 years ago

It must be an issue affecting only specific areas - Google Analytics shows that people are visiting and using the website as we speak

screen shot 2019-01-15 at 13 54 38

Screenshot taken at 13:54 on 15 January 2019

deniseOme commented 5 years ago

@mkarmona is looking into this. Apparently it is a "rate limit from the API code side, which should be removed".

deniseOme commented 5 years ago

Should be back to normal @pieterlukasse. Let us know how you get on.

pieterlukasse commented 5 years ago

is working again :+1: What was the issue?

deniseOme commented 5 years ago

@afaulconbridge has noticed a traffic spike of requests with header user Agent: "Open Targets Python Client/3.1.0".

@mkarmona has redeployed a new version without auth and rate limiting. It seems to work but do let us know if it misbehaves in the future.

pieterlukasse commented 5 years ago

ok, thanks!

afaulconbridge commented 5 years ago

To expand a bit on @deniseOme diagnosis, it seems that a user was using the Open Targets Python client to script against the API at about 200 requests per second for several minutes. In turn, this miss-triggered old partially disabled code relating to authentication and rate limiting (see http://blog.opentargets.org/2017/11/06/discovering-drug-targets-gets-faster-our-api-upgrades-to-v3/ ) which effectively shut out all users. This was because it had been disabled by making all access belong to a single user and giving that user a very high rate limit (but clearly one that wasn't high enough!).

To solve it, @mkarmona cleaned up and removed any of the old authentication code - which is really what should have happened in the first place!

pieterlukasse commented 5 years ago

thanks @afaulconbridge . Is there a link to the PR that fixes this? I have this issue locally and am wondering whether I could cherry-pick the fix.