Python access leads to IP blacklisting?

[mgraauw]

After running the sample Python script at: https://github.com/IHTSDO/SNOMED-in-5-minutes/tree/master/python3-examples yesterday, I was blocked from accessing not only the public Snowstorm, but the regular browser as well. After some 24 hrs it worked, tried to access the FHIR API instead, same result.

I found this ticket: https://github.com/IHTSDO/snowstorm/issues/539, but changing the user Agent does not help: apparently not only the User Agent 'Python3' is blocked, but the IP address using it is now blocked as well. Not sure whether the IP blacklisting ends after some 24 hrs or it was manually removed from some blacklist - I sent a mail to techsupport@snomed.org , got no answer but maybe it was followed up.

So should I wait 24 hrs or mail techsupport to have my IP unblocked?

[kaicode]

The public Snowstorm (snowstorm.ihtsdotools.org) and the browser (browser.ihtsdotools.org) are the same instance of Snowstorm. If you are blocked from one you are blocked from both.

My understanding is that using the default Python user agent request header causes the public Snowstorm to not respond or give some error code. Making just a couple of requests in this way should not cause IP blocking. The blocking should only happen due to the volume of requests exceeding the rate limit. Any IP that exceeds the request rate limit will be blocked for 24 hours.

The SNOMED in 5 minutes project is leading to users being blocked. We should add some advice there about adding contact details to the user agent request. This way tech services should be able to contact users who are causing service stability problems.

FYI - Snowstorm Lite is a fast and lightweight alternative that works for most SNOMED CT FHIR use cases https://github.com/IHTSDO/snowstorm-lite

[kaicode]

Your IP should be unblocked after 24 hours. Feel free to mail techsupport but it may take several hours to process your request.

[kaicode]

Really we recommend running your own Snowstorm or Snowstorm Lite instance to support scripting.

There is an unofficial demo instance of Snowstorm Lite here at the moment if you would like to try that out https://snowstorm-lite.nw.r.appspot.com/fhir/CodeSystem

[mgraauw]

Thanks for your response. To be clear, I issued no more than a few requests - just the 5 minute script yesterday, and only 3 FHIR API calls for a single concept id today. Tested just now, with the IP from my mobile hotspot:

• browser works (twice)
• Postman works (twice) Running only this:

  import requests
  base = 'https://snowstorm.ihtsdotools.org/fhir/CodeSystem/$lookup?system=http://snomed.info/sct&code=74400008'
  resp = requests.get(base)
  resp.json()

  gives no reply, and afterwards both the browser and Postman timeout. Apparently a single call is enough.

I do appreciate that running my own Snowstorm is better, but of course I'd like to test if Snowstorm works for me before going that route - using the RF2 files is another feasible route. Thanks for the link to the other demo, that does work and is enough for now.

[kaicode]

The guidance has been updated on issue 539 and in the SNOMED In 5 Minutes python script.

[mgraauw]

Thanks, that works