Download progress for desktop zarr files (remote and local)

AllenInstitute / biofile-finder

An open-use web application created for easy access, collaboration, and sharing of datasets through rich metadata search, filter, sort, and direct viewing in common industry applications or in our web-based 3D Volume Viewer.

https://biofile-finder.allencell.org/

BSD 3-Clause "New" or "Revised" License

3 stars 1 forks source link

Download progress for desktop zarr files (remote and local) #236

Closed BrianWhitneyAI closed 2 months ago

BrianWhitneyAI commented 2 months ago

The current Zarr download method does not show the progress of the download. This is particularly confusing with multifile where subsequent lower files appear asynchronously.

BrianWhitneyAI commented 2 months ago

We have decided to forgo individual local file tracking for now. In the situation that we use a /allen path and want to download locally we should use the FmsFile class's logic for determining cloud paths for all zarrs so that the existing download tracking functions correctly. This might prove more complicated as right now we do not support download paths for paths like s3://allencell/