Reduce running time to improve TATs

eastgenomics / eggd_ici_uploader

MIT License

0 stars 0 forks source link

Reduce running time to improve TATs #6

Open RSWilson1 opened 1 week ago

RSWilson1 commented 1 week ago

Improve the running time currently 1.5-2 hours. Suggestion use dx-fuse instead of downloading and uploading.

spaul91 commented 6 days ago

Alternative suggestion: wait for ICI team to implement API support to fetch workflow file-structure config, then use that to target files to download. Currently the app fetches everything in a directory, which is the rate-limiting step