Adding data auth layer.

vanvalenlab / deepcell-tf

Deep Learning Library for Single Cell Analysis

https://deepcell.readthedocs.io

Other

413 stars 95 forks source link

Adding data auth layer. #678

Closed rossbar closed 1 year ago

rossbar commented 1 year ago

What

Adds Python user interface to authentication layer.

Ports over the example function from @alexanderpg96 for inclusion in the deepcell-tf library. This could arguably be included somewhere else (e.g. deepcell-toolbox) - I'm open to feedback here. There are still a few things to hammer out on the Python side (tests, default download locations, etc.) as well as with the deepcell hosting app (update URL, etc.)

@msschwartz21 WDYT about the general approach?

rossbar commented 1 year ago

FTR usage looks something like:

# Import location will be updated, only for testing at this stage
>>> from deepcell.datasets._auth import fetch_data
>>> fetch_data("tissuenet/tissuenet_v1.0.zip")

which will download tissuenet to ~/.deepcell/tissuenet_v1.0.zip[^1]

[^1]: feedback on preferred default download location welcome - where are models/data currently stored by default?

msschwartz21 commented 1 year ago

Just wanted to drop a link to the source code for the keras function that we have been using in case we want to borrow anything regarding extracting archives or verifying hashes

https://github.com/keras-team/keras/blob/ab566fdc9ae83a88dc4bf149e055773e4d9edf86/keras/utils/data_utils.py#L194