SPIKE: Facilities API: 5-min spikes in 503 errors on CCP - Githubissues

department-of-veterans-affairs / va.gov-cms

Editor-centered management for Veteran-centered content.

https://prod.cms.va.gov

GNU General Public License v2.0

97 stars 69 forks source link

SPIKE: Facilities API: 5-min spikes in 503 errors on CCP #19402

Open jilladams opened 1 week ago

jilladams commented 1 week ago

Description or Additional Context

Lately, we get ~daily warning or alarms in Datadog, for 5 minute windows when the Facilities API returns a spike of 503 errors.

This is in calls to the PPMS Community Care Providers endpoint, and we receive back:

<html><body><h1>503 Service Unavailable</h1>
No server is available to handle this request.
</body></html>

We'd like to figure out what these windows correlate to, if it's preventable, or if we need to escalate to PPMS.

Acceptance Criteria

[ ] Determine root cause, if possible
[ ] Determine if we need to reach out to PPMS for support

jilladams commented 4 days ago

SNOW ticket filed: INC35601407