search

fedspendingtransparency / usaspending-api

Server application to serve U.S. federal spending data via a RESTful API
https://www.usaspending.gov
Creative Commons Zero v1.0 Universal
309 stars 112 forks source link

[DEV-11409] Grafana Migration Part 2 #4208

Closed ayubshahab closed 1 week ago

ayubshahab commented 1 month ago

Description: This ticket/PR was created to continue the Grafana Migration from DataDog. Specifically, we addressed the issue of subprocess tracing inside the bulk_downloader and further streamlined alot of the information we were capturing. Logs inside loki and traces inside tempo have also been aligned. The testing suite was also updated. This PR contains the initial changes implemented inside of DEV-11215.

NOTE: This PR was supposed to go into the main feature Branch DEV-11215, however since we are no longer tracking that. The changes from this branch DEV-11409 will get directly merged into QAT and other environments. Therefore this PR is getting closed and another PR for that is up. The link to that can be found here.

Technical details: Since, Otel does not support filtering of traces and spans, there is some left over code that was being utilized with Datadog. That code has been left inside the tracing file in case of future uses but can be safely taken out if need be.

Additional Details: Previously, we were able to perform some foundational work regarding the Grafana Migration from Datadog. Baseline functionality was implemented so UsaSpending API endpoints could be traced and relevant information such as traces could be ingested in Tempo and logging could be ingested in Loki. This goal was accomplished with DEV-11215.

Requirements for PR merge:

  1. [x] Unit & integration tests updated
  2. [x] API documentation updated
  3. [x] Necessary PR reviewers:
    • [x] Backend
  4. [x] Appropriate Operations ticket(s) created
  5. [x] Jira Ticket DEV-123:
    • [x] Link to this Pull-Request

Area for explaining above N/A when needed:

ayubshahab commented 1 week ago

This PR was supposed to go into the main feature Branch DEV-11215, however since we are no longer tracking that respective branch. The changes from this branch DEV-11409 will get directly merged into QAT and other environments. Therefore this PR is getting closed and another PR for that is up. The link to that can be found https://github.com/fedspendingtransparency/usaspending-api/pull/4221.