Monitor and Flag Data Storage Status in ComposeDB

[nutrina]

User Story:

As a Developer or Data Analyst working on Passport or Scorer applications, I want a reliable way to monitor if data is properly saved in Compose-DB, So that I can quickly identify data loss or discrepancies for further analysis and action.

Acceptance Criteria

Scenario 1: GIVEN I am a Developer or Data Analyst WHEN I query the CeramicCache model THEN I should see the following nullable columns compose_db_save_status and compose_db_stream_id AND they should show compose_db_save_status="saved" and compose_db_stream_id="<valid stream id>" if the stamp was successfully stored in Compose-DB AND it should indicate compose_db_save_status="failed" and compose_db_stream_id=null if the stamp was not successfully stored in Compose-DB

Scenario 2: GIVEN I am a user of the Passport app WHEN I claim a stamp (new or refreshed) AND the stamp is saved to the CeramicCache model THEN the compose_db_save_status field should be reset to null AND the compose_db_stream_id field should be left unchanged (will be null by default)

Scenario 3: GIVEN I am a user of the Passport app WHEN I claim a stamp (new or refreshed) AND the saving to Compose-DB has been performed THEN the compose_db_save_status field in CeramicCache should reflect the success or failure of the saving action AND the compose_db_stream_id field should store the valid stream_id

Tech Details:

• Update the CeramicCache model to include the status column.
• Reset status to null whenever a stamp is saved or updated.
• Create an API to set the status field.
• Update the Passport App to call this new service.
• Store the Ceramic stream ID when we receive confirmation that the data was written to Ceramic

Open Questions:

• Should we store additional troubleshooting information from Compose-DB?
• Instead of resetting status to null, should we set it to "outdated" to indicate a pending save to Compose-DB?
• Should it be a separate endpoint?

Notes/Assumptions:

This feature is not about the data storage mechanism itself, but rather aims to monitor and flag the reliability of data storage in Compose-DB.

[Jkd-eth]

@nutrina Per the conversation with Ceramic - Jan 10th - Let's include a new column to include the ceramic stream ID so that we can ensure there are no data discrepancies between Ceramic & Passport (or at least be able to understand them)