marlon360 / rki-covid-api

🦠🇩🇪📈 An API for the spread of covid-19 in Germany. Data from Robert-Koch-Institut.
https://api.corona-zahlen.org/
Creative Commons Attribution 4.0 International
250 stars 50 forks source link

503 Service Temporarily Unavailable #68

Closed nimpla closed 3 years ago

nimpla commented 3 years ago

Hello, the nginx server gives "503 Service Temporarily Unavailable" as response.

helmar74 commented 3 years ago

same here, when contacting: https://api.corona-zahlen.org/germany

marlon360 commented 3 years ago

I had to take it offline for today. The RKI API responds with errors on some endpoints. This causes a bypass of the caching system which leads to many requests to the RKI API. Last time this happened my IP got blocked by their server. To prevent this to happen again, I have to add an error handler that caches the error.

This API gets 6 million hits per day, which looks like a DDoS attack if not cached correctly.

This endpoint for example responds with a time out error: https://services7.arcgis.com/mOBPykOjAyBO2ZKk/arcgis/rest/services/RKI_COVID19/FeatureServer/0/query?where=NeuGenesen+IN(1,0)&objectIds=&time=&resultType=standard&outFields=AnzahlGenesen,MeldeDatum,IdBundeland&returnIdsOnly=false&returnUniqueIdsOnly=false&returnCountOnly=false&returnDistinctValues=false&cacheHint=false&orderByFields=IdBundesland&groupByFieldsForStatistics=IdBundesland&outStatistics=[{%22statisticType%22:%22sum%22,%22onStatisticField%22:%22AnzahlGenesen%22,%22outStatisticFieldName%22:%22recovered%22},{%22statisticType%22:%22max%22,%22onStatisticField%22:%22MeldeDatum%22,%22outStatisticFieldName%22:%22date%22}]&having=&resultOffset=&resultRecordCount=&sqlFormat=none&f=pjson&token=

helmar74 commented 3 years ago

I had to take it offline for today. The RKI API responds with errors on some endpoints. This causes a bypass of the caching system which leads to many requests to the RKI API. Last time this happened my IP got blocked by their server. To prevent this to happen again, I have to add an error handler that caches the error.

This API gets 6 million hits per day, which looks like a DDoS attack if not cached correctly.

This endpoint for example responds with a time out error: https://services7.arcgis.com/mOBPykOjAyBO2ZKk/arcgis/rest/services/RKI_COVID19/FeatureServer/0/query?where=NeuGenesen+IN(1,0)&objectIds=&time=&resultType=standard&outFields=AnzahlGenesen,MeldeDatum,IdBundeland&returnIdsOnly=false&returnUniqueIdsOnly=false&returnCountOnly=false&returnDistinctValues=false&cacheHint=false&orderByFields=IdBundesland&groupByFieldsForStatistics=IdBundesland&outStatistics=[{%22statisticType%22:%22sum%22,%22onStatisticField%22:%22AnzahlGenesen%22,%22outStatisticFieldName%22:%22recovered%22},{%22statisticType%22:%22max%22,%22onStatisticField%22:%22MeldeDatum%22,%22outStatisticFieldName%22:%22date%22}]&having=&resultOffset=&resultRecordCount=&sqlFormat=none&f=pjson&token=

Thanks for the info. Will it be available again tomorrow?

Thank you for your good work 😀

marlon360 commented 3 years ago

The RKI fixed their API and this server is back online. I added error handling. When the RKI API has problems you will receive a response like this:

{
  "error": {
    "message": "There is a problem with the official RKI API.",
    "rkiError": {
      "code": 503,
      "message": "Timeout Error",
      "details": []
    },
    "url": "https://services7.arcgis.com/mOBPykOjAyBO2ZKk/arcgis/rest/services/RKI_COVID19/FeatureServer/0/query?where=NeuGenesen IN(1,0,-9) AND MeldeDatum >= TIMESTAMP '2021-01-11' AND IdBundesland = 2&objectIds=&time=&resultType=standard&outFields=AnzahlGenesen,MeldeDatum,Bundesland,IdBundesland&returnIdsOnly=false&returnUniqueIdsOnly=false&returnCountOnly=false&returnDistinctValues=false&cacheHint=false&orderByFields=IdBundesland,MeldeDatum&groupByFieldsForStatistics=IdBundesland,MeldeDatum,Bundesland&outStatistics=[{\"statisticType\":\"sum\",\"onStatisticField\":\"AnzahlGenesen\",\"outStatisticFieldName\":\"recovered\"}]&having=&resultOffset=&resultRecordCount=&sqlFormat=none&f=pjson&token="
  }
}