AllenNeuralDynamics / aind-dynamic-foraging-data-utils

Tools for the analysis of behavior and neural data from the dynamic foraging task
MIT License
2 stars 0 forks source link

Easy tool to download nwb file #27

Open alexpiet opened 4 months ago

alexpiet commented 4 months ago

@rachelstephlee and I imagine a workflow that we download a handful of NWB files to develop packages locally

We would like a programmatic way to do this, with flags for separating video, or other large attributes.

rachelstephlee commented 1 month ago

@faezeamin also wants this-- and she mentioned she'd want to know if there are videos.

David gave me this API for programmatically attaching NWB data-assetes

https://docs.codeocean.com/user-guide/v2.22.0/code-ocean-api/capsule#attach-data-assets

each request can only take about a list of 100 data assets. for questions, ask david feng.

sample code from the link.

from codeocean.data_asset import DataAssetAttachParams

data_assets = [
    DataAssetAttachParams(id="1fa0c990-3b5c-402f-ab3c-00cac6eed21e"),
    DataAssetAttachParams(id="1az0c240-1a9z-192b-pa4c-22bac5ffa17b", mount="Reference"),
]        

results = client.capsules.attach_data_assets(
    capsule_id="zv082356-a032-1b97-90b0-f209b8728927",
    attach_params=data_assets,
)

Passed this to @ulisespereira bc he expressed interest to help + wanted data.

rachelstephlee commented 5 days ago

@irisstone fixed this, giving us this script. you can do at most 100 at a time.

data_assets = generate_data_asset_params(data_asset_IDs, mount_point=None)
token = os.getenv('DATA_ASSETS_KEY')
client = CodeOcean(domain="https://codeocean.allenneuraldynamics.org", token=token)
capsule_id = "3f575b5f-5b29-4e09-b400-630a2a44b564"
results = client.capsules.attach_data_assets(    capsule_id=capsule_id,    attach_params=data_assets,)

and then everything attaches like this:

image

This needs to be made into a function, then put into a PR.

rachelstephlee commented 3 days ago

@ZhixiaoSu figuerd out that there was around a 200 session limit to code ocean. (11/27/2024)

TODO for Sue:

  1. share the capsule that broke/share the length of data assets
  2. share the workaround code you used to pull NWB information out to load locally.
  3. Rachel will want to follow up with david feng on new limits