get data into ceph - Githubissues

aicoe-kaggle / diabetic-retinopathy

Other

0 stars 0 forks source link

get data into ceph #4

Open MichaelClifford opened 3 years ago

MichaelClifford commented 3 years ago

download data from kaggle and store in ceph- create private bucket

Unzip and store data in persistent storage. Need access controls. Only participants who have formally accepted kaggle competition conditions can have access to data. Data can't be used for any other (non-competition) purposes.

[ ] include reusable script for data connection

pacospace commented 3 years ago

We can use the template issue: https://github.com/operate-first/support/issues/new?assignees=&labels=user-support&template=ceph_bucket_request.md&title=, we might need at least 200GB I would say, if we consider all dataset and future processed data, models we want store, wdyt?

MichaelClifford commented 3 years ago

see this issue for updates on bucket https://github.com/operate-first/support/issues/287