severinbeauvais
commented
2 months ago Impact of problem 1: Some documents are uploaded to Minio but we've lost the keys. This will not cause corrupt data but we will have orphaned documents, which take up space but we can't download (because we don't know about them).
Impact of problem 2: Some documents are deleted from Minio but their keys are still in a filing. Vysakh says the Legal API will verify that the documents matching those keys exist, otherwise it will reject the filing. This will not cause corrupt data but it will be confusing for the user, who previously cleared/deleted a document (but didn't save the draft) and now needs to clear it from their filing again before submitting again.
Work required:
- suggest uploading files through Legal API so that it can keep track of keys (linked to a draft filing)
- need new Legal API documents upload endpoint (which gets a Minio key and uploads the doc to Minio)
- need to update document handling (probably using existing "documents" table) to keep track of added/removed files; this will impact several filing types
- need to update document handling to package the documents into filings when the filings are submitted
- need to delete documents when filings are deleted (may exist)
- need to update Create UI (several filings) and Edit UI (coop special resolution) to handle documents differently (possibly not including them in the filing JSON and restoring them separately)
In the UIs, it's possible for users to upload a document to Minio and then "lose" the document key if the UI is closed before the key is saved (eg, navigate away, power interruption, etc). This results in orphaned Minio documents.
This ticket is to design and implement a way for the Legal API to keep track of document keys for upload Minio files, and then auto-delete those Minio files if the document keys are not saved in a filing JSON.
There's a second issue that needs to be solved by design: what if the user clears (deletes) a document on their filing, and they don't save the filing, and later they reload the filing? The document will be on the filing but will actually not exist in Minio.