search

18F / api.data.gov

A hosted, shared-service that provides an API key, analytics, and proxy solution for government web services.
https://api.data.gov
Other
97 stars 43 forks source link

[Developing] Possible partial outage #472

Closed gbinal closed 6 years ago

gbinal commented 6 years ago

There's intermittent reports of a partial AWS outage on twitter and we're investigating to see if there's any impact on the api.data.gov service. Will update this issue as we learn more.

gbinal commented 6 years ago

I'm testing a number of endpoints that are flowing through api.data.gov and they appear to be working.

https://api.gsa.gov/analytics/dap/v1/reports/download/data?api_key=DEMO_KEY1 https://api.nasa.gov/planetary/apod?api_key=DEMO_KEY&date=2018-10-14

This hopefully means that if there are any issues, they aren't affecting the performance of the APIs using the service.

gbinal commented 6 years ago

That said, I'm getting some odd errors when loading various pages in the admin backend. Trying to reproduce...

gbinal commented 6 years ago

The issues in the admin backend have appeared to gone away and it seems to be working now.

The two issues I was experiencing was an error message on https://api.data.gov/admin/#/stats/drilldown and then a different error (a popup message referring to the tables having an error) appearing on https://api.data.gov/admin/#/stats/users, along with no data then loading on the page.

I was able to reproduce those issues for about 5-10 minutes, but at the same time as still seeing all of the endpoints working.

Now, everything appears to be working again fine.

gbinal commented 6 years ago

There were no New Relic errors or emails from agencies reporting a problem throughout this period. It was just the twitter traffic reporting some AWS issues that got us looking. That said, there haven't been very many tweets, so if there was an issue, it must not have been very widespread.

gbinal commented 6 years ago

We're going to continue monitoring things but it appears that there's no problem with the api.data.gov service (and there possibly never was).

https://status.aws.amazon.com was just updated with a network connectivity issue for Amazon Elastic Compute Cloud (N. Virginia) that reads: 12:22 PM PDT We are investigating network connectivity issues for some instances within the US-EAST-1 Region..

gbinal commented 6 years ago

Just to follow up on this, it appears that any issues with data resolving in the admin panel was fleeting and did not recur. Also, api calls continued to go through nominally throughout.

I'm going to go ahead and close this issue.