search

NHMDenmark / DaSSCo-Integration

This Repo will include integration of dassco storage from northtec
0 stars 0 forks source link

Workflow issues ARS #32

Closed Baeist closed 4 months ago

Baeist commented 7 months ago
@bhsi-snm @k-zamzam Normal flow: Calls Asset status Comments/Issues/Needs
POST create asset METADATA_RECEIVED Fine
PUT upload file METADATA_RECEIVED Fine
POST synchronize erda ASSET_RECEIVED / COMPLETED Changes to COMPLETED once its been synchronized. If it fails enough times changes to ERDA_ERROR. Fine
GET asset status COMPLETED This always shows "share_allocation_mb": null
POST open share COMPLETED We need another status here. REOPENED perhaps.
GET list available files COMPLETED Links to the available files have a faulty url. It specifies /file_proxy/api twice.
GET get file COMPLETED Fine
POST synchronize erda ASSET_RECEIVED / COMPLETED Changes back to ASSET_RECEIVED here before COMPLETED once its been synchronized again. If it fails enough times changes to ERDA_ERROR. Fine
GET asset status COMPLETED Fine
PUT update asset COMPLETED Fine
ERDA_ERROR flow: Calls Asset status Comments/Issues/Needs
GET asset status ERDA_ERROR Assuming we get here after trying to sync with erda. Fine
POST synchronize erda ASSET_RECEIVED / ERDA_ERROR Changes back to ASSET_RECEIVED here before ERDA_ERROR once again. "error_message": "com.jcraft.jsch.JSchException: java.io.IOException: inputstream is closed"
GET asset status ERDA_ERROR Given enough tries it will eventually work again at some point and change to status COMPLETED (except for asset_18 presumably).

It generally seems that the more i work with an asset the higher the risk of the asset becoming inaccessible and just giving "no writeable directory found" errors despite declaring that a share is opened. I do not know exactly when it goes wrong here. With the information i can see it feels a bit random but could very well not be.

Baeist commented 6 months ago

@bhsi-snm @k-zamzam Used synchronize erda 5 times and after 6-7 mins of having the status set at metadata received for all of them their final status ended up as ERDA_ERROR but they had different error messages. This was also the second time in a row i tried syncng all 5 i did not look at the error messages from the first try though. This is from the Get in progress assets : { "asset_guid": "test-asset-guid-5", "parent_guid": null, "error_timestamp": "2024-03-06T13:55:24.894Z", "status": "ERDA_ERROR", "error_message": "com.jcraft.jsch.JSchException: java.io.IOException: inputstream is closed", "share_allocation_mb": 1 }, { "asset_guid": "test-asset-guid-4", "parent_guid": null, "error_timestamp": "2024-03-06T13:55:25.161Z", "status": "ERDA_ERROR", "error_message": "com.jcraft.jsch.JSchException: java.net.ConnectException: Connection refused", "share_allocation_mb": 1 }, { "asset_guid": "test-asset-guid-3", "parent_guid": null, "error_timestamp": "2024-03-06T13:55:25.347Z", "status": "ERDA_ERROR", "error_message": "com.jcraft.jsch.JSchException: java.net.ConnectException: Operation timed out", "share_allocation_mb": 1 }, { "asset_guid": "test-asset-guid-2", "parent_guid": null, "error_timestamp": "2024-03-06T13:55:25.374Z", "status": "ERDA_ERROR", "error_message": "com.jcraft.jsch.JSchException: java.net.ConnectException: Operation timed out", "share_allocation_mb": 1 }, { "asset_guid": "test-asset-guid-1", "parent_guid": null, "error_timestamp": "2024-03-06T13:55:25.574Z", "status": "ERDA_ERROR", "error_message": "com.jcraft.jsch.JSchException: java.io.IOException: inputstream is closed", "share_allocation_mb": 10 },

Baeist commented 6 months ago

Right now it feels like the issues can mostly be isolated to when a share is opened multiple times and sort of crashes everything that has to do with it. We need a status for when a share is open so we or someone else wont accidentally open more than one. We can keep track of when we have a share open but not if others also have access.