CDCgov / prime-reportstream

ReportStream is a public intermediary tool for delivery of data between different parts of the healthcare ecosystem.
https://reportstream.cdc.gov
Creative Commons Zero v1.0 Universal
64 stars 38 forks source link

Update Internal Blobstore Naming Strategy #14330

Open arnejduranovic opened 1 week ago

arnejduranovic commented 1 week ago

User Story

As a blob in ReportStream's Storage Container, I want to ensure my name does not collide with any other blob name, so that the pipeline step that generates me does not fail and throw and exception.

Description/Use Case

Each step in the universal pipeline generates one or more reports. These reports are then stored in the proper folder in the Azure Storage Container. The blobs are named with complex naming logic that may cause collisions and lead to the upload to fail and the pipeline step to then fail as well.

There are two important features in ReportStream concerning filenames:

  1. ReportStream allows receivers to specify the format of their filename and what info their filename contains (see prime-router/metadata/file_name_templates/file-name-templates.yml)
  2. ReportStream allows senders to specify the name of the submitted report in the header of their submission request (payloadName). If payloadName is specified AND the topic has isSendOriginal enabled, then the receiver's filename convention should be ignored and the name of the blob should equal: reportID + payloadName.

The above two use cases shall still be supported, but whatever the filename a receiver will receive, it shall not be the name of the blob in ReportStream's internal blob store. ReportStream's internal blob store shall only include the report's UUID in the name and nothing else. This is to avoid potential naming collisions.

Risks/Impacts/Considerations

Dev Notes

Acceptance Criteria

Andrey-Glazkv commented 6 days ago

Hey team! Please add your planning poker estimate with Zenhub @adegolier @arnejduranovic @brick-green @jack-h-wang @jalbinson @mkalish @thetaurean @david-navapbc @JFisk42