search

int-brain-lab / iblenv

Unified environment and Issue tracker for all IBL
MIT License
10 stars 10 forks source link

[Bug report] - HTTP Forbidden error when loading repro-ephys data from OpenALyx #313

Closed anne-urai closed 2 years ago

anne-urai commented 2 years ago

Describe the bug

Following the instructions here https://int-brain-lab.github.io/iblenv/notebooks_external/data_release_repro_ephys.html#Overview-of-the-Data

pw = 'international'
one = ONE().setup(base_url='https://openalyx.internationalbrainlab.org')
# password: international

# Find sessions that have data and are tagged for the repeated site paper
rep_site_sessions = one.alyx.rest('sessions', 'list', 
                                  dataset_types='spikes.times',
                                  tag='2022_Q2_IBL_et_al_RepeatedSite')

and then downloading the ALF files for one example dataset,

#%% test if I can indeed download this
files = one.load_collection(rep_site_sessions[-1], 'alf', download_only=True)

I can download a subset of the files, but not all (it ends in a HTTP forbidden error). Across a few tested datasets, they are usually the same npy files that have this error (o.a. trials.probabilityLeft, trials.choice).

Since I can download many other files correctly, and I'm connected to OpenAlyx correctly, could there be another reason why I can't access a subset of files?

Note: I get the same HTTP: Forbidden errors when loading through trials = one.load_object(rep_site_sessions[-1], 'trials', collection='alf').

Full output:

Connected to https://openalyx.internationalbrainlab.org as intbrainlab
Downloading: /Users/urai/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/probes.description.091b2d8b-0a05-45d8-a396-a697d253410c.json Bytes: 408
Downloading: /Users/urai/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/_ibl_rightCamera.times.1702ec5e-ccd0-4ba1-858f-772f17838be4.npy Bytes: 5327272
Downloading: /Users/urai/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/_ibl_leftCamera.dlc.096e536d-f0e4-4532-8975-3fc1b9daa788.pqt Bytes: 64731877
Downloading: /Users/urai/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/rightCamera.ROIMotionEnergy.1de9959c-d03c-4045-971d-44d11e7a7812.npy Bytes: 8302880

  0%|          | 0/0.00038909912109375 [00:00<?, ?it/s]

  0%|          | 0/7.918243408203125 [00:00<?, ?it/s]

100%|██████████| 0.00038909912109375/0.00038909912109375 [00:00<00:00, 32.30s/it]
HTTP Error 403: Forbidden https://ibl.flatironinstitute.org/public/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/_ibl_trials.choice.27d1d25a-a801-4669-8c85-8cf4d2aea210.npy
Downloading: /Users/urai/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/bodyCamera.ROIMotionEnergy.41aff0bc-6e9c-4d08-9da5-8a63d092650e.npy Bytes: 1665128
100%|██████████| 1.5879898071289062/1.5879898071289062 [00:02<00:00,  1.32s/it]

 79%|███████▊  | 4.0/5.080482482910156 [00:03<00:01,  1.04it/s]

 51%|█████     | 4.0/7.918243408203125 [00:03<00:03,  1.03it/s]

  6%|▋         | 4.0/61.73312854766846 [00:04<00:58,  1.02s/it]Downloading: /Users/urai/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/_ibl_rightCamera.dlc.421e2377-b7f1-499f-8520-0f51b6da5236.pqt Bytes: 135561017
  0%|          | 0/129.28106021881104 [00:00<?, ?it/s]
100%|██████████| 5.080482482910156/5.080482482910156 [00:04<00:00,  1.11it/s]
HTTP Error 403: Forbidden https://ibl.flatironinstitute.org/public/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/_ibl_trials.firstMovement_times.4a7d46a3-ddcb-4218-bab8-c519b068ce3c.npy
HTTP Error 403: Forbidden https://ibl.flatironinstitute.org/public/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/_ibl_trials.probabilityLeft.4ba4bd8f-0a96-464b-8ac9-d22cc98b8b96.npy

100%|██████████| 7.918243408203125/7.918243408203125 [00:06<00:00,  1.21it/s]

 13%|█▎        | 8.0/61.73312854766846 [00:06<00:42,  1.26it/s]Downloading: /Users/urai/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/_ibl_trials.intervals_bpod.52006bb0-dee1-4bcf-9492-6fcbcc8f40db.npy Bytes: 13728

  0%|          | 0/0.013092041015625 [00:00<?, ?it/s]
100%|██████████| 0.013092041015625/0.013092041015625 [00:00<00:00,  9.95s/it]
Downloading: /Users/urai/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/_ibl_leftCamera.features.52e71104-8b63-4914-b986-c278fd86d762.pqt Bytes: 264201

  0%|          | 0/0.25196170806884766 [00:00<?, ?it/s]Downloading: /Users/urai/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/_ibl_rightCamera.features.55771c6a-d556-4d48-a87b-66031c76a6bf.pqt Bytes: 16253677

  0%|          | 0/15.500714302062988 [00:00<?, ?it/s]
100%|██████████| 0.25196170806884766/0.25196170806884766 [00:00<00:00,  3.26s/it]
  3%|▎         | 4.0/129.28106021881104 [00:04<02:10,  1.04s/it]HTTP Error 403: Forbidden https://ibl.flatironinstitute.org/public/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/_ibl_trials.response_times.58361f14-5611-4df0-8a26-91c8cc1595f9.npy

 19%|█▉        | 12.0/61.73312854766846 [00:08<00:33,  1.47it/s]Downloading: /Users/urai/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/_ibl_leftCamera.times.606c3d1b-02be-4ba6-8785-b908055c1c7d.npy Bytes: 2132904

  0%|          | 0/2.0340957641601562 [00:00<?, ?it/s]

  6%|▌         | 8.0/129.28106021881104 [00:07<01:45,  1.15it/s]

 26%|██▌       | 4.0/15.500714302062988 [00:04<00:12,  1.08s/it]
100%|██████████| 2.0340957641601562/2.0340957641601562 [00:02<00:00,  1.34s/it]
HTTP Error 403: Forbidden https://ibl.flatironinstitute.org/public/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/_ibl_trials.contrastLeft.62876e88-21cf-4211-a937-c204601dbf23.npy

 32%|███▏      | 20.0/61.73312854766846 [00:12<00:23,  1.75it/s]Downloading: /Users/urai/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/_ibl_trials.stimOff_times.69713007-018f-4cce-875e-29a90c523bd6.npy Bytes: 6928

100%|██████████| 0.0066070556640625/0.0066070556640625 [00:00<00:00,  1.74it/s]
  9%|▉         | 12.0/129.28106021881104 [00:09<01:32,  1.27it/s]HTTP Error 403: Forbidden https://ibl.flatironinstitute.org/public/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/_ibl_trials.stimOn_times.7316e643-c39c-47b4-b5ca-49ba932ec536.npy

 52%|█████▏    | 8.0/15.500714302062988 [00:06<00:06,  1.19it/s]

 39%|███▉      | 24.0/61.73312854766846 [00:14<00:19,  1.90it/s]Downloading: /Users/urai/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/rightROIMotionEnergy.position.75c5f961-863c-4838-99ee-e149db357db2.npy Bytes: 160

100%|██████████| 0.000152587890625/0.000152587890625 [00:00<00:00, 23.34s/it]
Downloading: /Users/urai/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/_ibl_bodyCamera.dlc.76e9daef-bc88-48ec-90bc-979b01ec0692.pqt Bytes: 3912445

  0%|          | 0/3.731198310852051 [00:00<?, ?it/s]

 12%|█▏        | 16.0/129.28106021881104 [00:12<01:19,  1.42it/s]

 77%|███████▋  | 12.0/15.500714302062988 [00:09<00:02,  1.35it/s]

 15%|█▌        | 20.0/129.28106021881104 [00:14<01:07,  1.63it/s]

 58%|█████▊    | 36.0/61.73312854766846 [00:18<00:10,  2.54it/s]

100%|██████████| 15.500714302062988/15.500714302062988 [00:11<00:00,  1.31it/s]
HTTP Error 403: Forbidden https://ibl.flatironinstitute.org/public/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/_ibl_trials.feedback_times.7910dcae-bba6-4d5f-a059-bb71fd92e3be.npy

 65%|██████▍   | 40.0/61.73312854766846 [00:20<00:08,  2.58it/s]
100%|██████████| 3.731198310852051/3.731198310852051 [00:04<00:00,  1.22s/it]
 19%|█▊        | 24.0/129.28106021881104 [00:16<01:03,  1.67it/s]HTTP Error 403: Forbidden https://ibl.flatironinstitute.org/public/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/_ibl_trials.intervals.894be4e2-be68-44a2-91aa-d8b14bb772ee.npy
Downloading: /Users/urai/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/leftROIMotionEnergy.position.8b09eeec-f537-4bf4-a85f-c321af1ec48a.npy Bytes: 160

100%|██████████| 0.000152587890625/0.000152587890625 [00:00<00:00, 26.31s/it]
Downloading: /Users/urai/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/_ibl_wheel.timestamps.a968cc34-2690-462f-bb04-41e449678690.npy Bytes: 11835936

  0%|          | 0/11.287628173828125 [00:00<?, ?it/s]

 71%|███████▏  | 44.0/61.73312854766846 [00:21<00:06,  2.72it/s]HTTP Error 403: Forbidden https://ibl.flatironinstitute.org/public/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/_ibl_trials.feedbackType.b24f0b45-7bf8-4f55-b242-89988e7b0dd7.npy
 22%|██▏       | 28.0/129.28106021881104 [00:18<00:56,  1.80it/s]Downloading: /Users/urai/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/_ibl_wheelMoves.peakAmplitude.b7d6a913-63b8-4d0a-9316-b2963ba9f7e9.npy Bytes: 8616

100%|██████████| 0.00821685791015625/0.00821685791015625 [00:00<00:00,  2.10it/s]

 78%|███████▊  | 48.0/61.73312854766846 [00:22<00:04,  2.87it/s]Downloading: /Users/urai/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/bodyROIMotionEnergy.position.b8318642-3c72-4b47-a2fa-22b963814c68.npy Bytes: 160

100%|██████████| 0.000152587890625/0.000152587890625 [00:00<00:00, 21.85s/it]
Downloading: /Users/urai/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/_ibl_trials.itiDuration.c5cc2fc5-01e5-4b7b-b483-6ece5cc384b2.npy Bytes: 6928

100%|██████████| 0.0066070556640625/0.0066070556640625 [00:00<00:00,  2.07it/s]
 25%|██▍       | 32.0/129.28106021881104 [00:19<00:48,  1.99it/s]

 84%|████████▍ | 52.0/61.73312854766846 [00:23<00:03,  3.01it/s]Downloading: /Users/urai/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/_ibl_bodyCamera.times.c6cc628f-489a-44f2-bb76-f4ab10d5fbaf.npy Bytes: 1068432

  0%|          | 0/1.0189361572265625 [00:00<?, ?it/s]

 28%|██▊       | 36.0/129.28106021881104 [00:21<00:45,  2.05it/s]
 35%|███▌      | 4.0/11.287628173828125 [00:04<00:08,  1.12s/it]

 97%|█████████▋| 60.0/61.73312854766846 [00:26<00:00,  3.28it/s]

100%|██████████| 1.0189361572265625/1.0189361572265625 [00:01<00:00,  1.59s/it]

100%|██████████| 61.73312854766846/61.73312854766846 [00:26<00:00,  2.33it/s]
Downloading: /Users/urai/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/_ibl_trials.goCueTrigger_times.de3c64ec-18fe-4037-99de-8715b2f77f90.npy Bytes: 6928

100%|██████████| 0.0066070556640625/0.0066070556640625 [00:00<00:00,  1.94it/s]
HTTP Error 403: Forbidden https://ibl.flatironinstitute.org/public/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/_ibl_trials.rewardVolume.deef1c1a-d5ea-42bc-a4de-3f7af0919115.npy
 31%|███       | 40.0/129.28106021881104 [00:23<00:40,  2.19it/s]HTTP Error 403: Forbidden https://ibl.flatironinstitute.org/public/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/_ibl_trials.goCue_times.e69e7741-c8f2-4394-aba9-b956c3c422f0.npy
Downloading: /Users/urai/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/_ibl_wheel.position.ef1c7a01-0cd1-440c-bd3e-ef9c0b4ef668.npy Bytes: 11835936

  0%|          | 0/11.287628173828125 [00:00<?, ?it/s]Downloading: /Users/urai/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/_ibl_wheelMoves.intervals.f1fe48d7-0f6b-4d53-8d79-62eed427cade.npy Bytes: 17104

  0%|          | 0/0.0163116455078125 [00:00<?, ?it/s]

100%|██████████| 0.0163116455078125/0.0163116455078125 [00:00<00:00,  8.55s/it]
 34%|███▍      | 44.0/129.28106021881104 [00:24<00:36,  2.35it/s]
 71%|███████   | 8.0/11.287628173828125 [00:07<00:02,  1.11it/s]HTTP Error 403: Forbidden https://ibl.flatironinstitute.org/public/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/_ibl_trials.contrastRight.f91b4334-79fe-45b7-97b7-d82127bc778a.npy
Downloading: /Users/urai/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019-12-03/001/alf/leftCamera.ROIMotionEnergy.f95ac329-79a7-44da-b337-a0f3f434ea0a.npy Bytes: 3324192

 37%|███▋      | 48.0/129.28106021881104 [00:25<00:33,  2.45it/s]
100%|██████████| 11.287628173828125/11.287628173828125 [00:09<00:00,  1.20it/s]
 40%|████      | 52.0/129.28106021881104 [00:27<00:30,  2.50it/s]

 43%|████▎     | 56.0/129.28106021881104 [00:28<00:27,  2.67it/s]

100%|██████████| 3.170196533203125/3.170196533203125 [00:03<00:00,  1.18s/it]
 50%|████▉     | 64.0/129.28106021881104 [00:31<00:21,  3.02it/s]

 56%|█████▌    | 72.0/129.28106021881104 [00:33<00:17,  3.34it/s]

100%|██████████| 11.287628173828125/11.287628173828125 [00:09<00:00,  1.18it/s]
100%|██████████| 129.28106021881104/129.28106021881104 [00:42<00:00,  3.01it/s]
Traceback (most recent call last):

  File "/anaconda3/envs/iblenv/lib/python3.9/site-packages/spyder_kernels/py3compat.py", line 356, in compat_exec
    exec(code, globals, locals)

  File "/Users/urai/Documents/code/int-brain-lab/mouse_aging/load_public_repro_ephys.py", line 24, in <module>
    files = one.load_collection(rep_site_sessions[-1], 'alf', download_only=True)

  File "/anaconda3/envs/iblenv/lib/python3.9/site-packages/one/util.py", line 168, in wrapper
    return method(self, *args, **kwargs)

  File "/anaconda3/envs/iblenv/lib/python3.9/site-packages/one/util.py", line 152, in wrapper
    return method(self, eid, *args, **kwargs)

  File "/anaconda3/envs/iblenv/lib/python3.9/site-packages/one/api.py", line 1242, in load_collection
    files = self._check_filesystem(datasets, offline=offline)

  File "/anaconda3/envs/iblenv/lib/python3.9/site-packages/one/api.py", line 532, in _check_filesystem
    new_files = self._download_datasets(dsets_to_download, update_cache=update_exists)

  File "/anaconda3/envs/iblenv/lib/python3.9/site-packages/one/api.py", line 1661, in _download_datasets
    return self._download_dataset(dsets, **kwargs)

  File "/anaconda3/envs/iblenv/lib/python3.9/site-packages/one/api.py", line 1819, in _download_dataset
    files = self._download_file(valid_urls, target_dir, **kwargs)

  File "/anaconda3/envs/iblenv/lib/python3.9/site-packages/one/api.py", line 1875, in _download_file
    local_path, md5 = self.alyx.download_file(url, target_dir=target_dir, return_md5=True)

  File "/anaconda3/envs/iblenv/lib/python3.9/site-packages/one/webclient.py", line 730, in download_file
    raise ex

  File "/anaconda3/envs/iblenv/lib/python3.9/site-packages/one/webclient.py", line 725, in download_file
    files = download_fcn(url, **pars)

  File "/anaconda3/envs/iblenv/lib/python3.9/site-packages/one/webclient.py", line 308, in http_download_file_list
    outputs.append(future.result())

  File "/anaconda3/envs/iblenv/lib/python3.9/concurrent/futures/_base.py", line 433, in result
    return self.__get_result()

  File "/anaconda3/envs/iblenv/lib/python3.9/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception

  File "/anaconda3/envs/iblenv/lib/python3.9/concurrent/futures/thread.py", line 52, in run
    result = self.fn(*self.args, **self.kwargs)

  File "/anaconda3/envs/iblenv/lib/python3.9/site-packages/one/webclient.py", line 393, in http_download_file
    raise e

  File "/anaconda3/envs/iblenv/lib/python3.9/site-packages/one/webclient.py", line 390, in http_download_file
    u = urllib.request.urlopen(req)

  File "/anaconda3/envs/iblenv/lib/python3.9/urllib/request.py", line 214, in urlopen
    return opener.open(url, data, timeout)

  File "/anaconda3/envs/iblenv/lib/python3.9/urllib/request.py", line 523, in open
    response = meth(req, response)

  File "/anaconda3/envs/iblenv/lib/python3.9/urllib/request.py", line 632, in http_response
    response = self.parent.error(

  File "/anaconda3/envs/iblenv/lib/python3.9/urllib/request.py", line 561, in error
    return self._call_chain(*args)

  File "/anaconda3/envs/iblenv/lib/python3.9/urllib/request.py", line 494, in _call_chain
    result = func(*args)

  File "/anaconda3/envs/iblenv/lib/python3.9/urllib/request.py", line 641, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)

HTTPError: Forbidden
micheleangelofabbri commented 2 years ago

Hey Anne, I was able to run your code above without any "HTTPError: Forbidden" errors. There was one, potentially important, change I made regarding the initial set up of openalyx. Unless you configured it earlier, it does not look like the password was set for your openalyx connection. Specifically I am referring to this part:

...
pw = 'international'
one = ONE().setup(base_url='https://openalyx.internationalbrainlab.org')
...

To be certain you do not have anything weird that might be causing problems in your stored ONE credentials please run the following command in your terminal: mv /Users/urai/.one /Users/urai/.one_bkup

Then please reattempt to load your data as before, but with the following difference:

from one.api import ONE
pw = 'international'
one = ONE(base_url='https://openalyx.internationalbrainlab.org', password=pw, silent=True)

Let me know if that works for you or if you run into the same issue.

Full list of commands that functioned for me and their output:

(ibl_py3.9) michele@gala:~/Documents$ python
Python 3.9.12 (main, Jun  1 2022, 11:38:51) 
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from one.api import ONE
>>> pw = 'international'
>>> one = ONE(base_url='https://openalyx.internationalbrainlab.org', password=pw, silent=True)
>>> rep_site_sessions = one.alyx.rest('sessions', 'list', dataset_types='spikes.times', tag='2022_Q2_IBL_et_al_RepeatedSite')
>>> files = one.load_collection(rep_site_sessions[-1], 'alf', download_only=True)
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
/home/michele/Downloads/ONE/openalyx.internationalbrainlab.org/wittenlab/Subjects/ibl_witten_13/2019
>>> trials = one.load_object(rep_site_sessions[-1], 'trials', collection='alf')
>>> type(trials)
<class 'one.alf.io.AlfBunch'>
...
k1o0 commented 2 years ago

@micheleangelofabbri I think this is entirely unrelated to ONE parameters. It appears to be a problem of file permissions on FlatIron. You are probably not seeing any download errors because you're using AWS instead of FlatIron. I can reproduce Anne's issue on my machine.

@anne-urai You can download from AWS instead of FlatIron by installing boto3 into your environment (pip install boto3). If you then restart python / your notebook, the downloads should work without error.

k1o0 commented 2 years ago

I can see that this error occurred because some trials datasets were removed but the public filesystem wasn't rebuilt. I'll try to get this done this week.

anne-urai commented 2 years ago

Thanks @k1o0, indeed installing boto3 does the trick. Perhaps this could be added to the ONE-api requirements?

k1o0 commented 2 years ago

Yes we'll add it in the next release