search

biobricks-ai / biobricks

BioBricks makes loading data from biological datasets and databases easy. Python and R interfaces, data version control, and an API for pulling datasets that have been converted to easy-to-use formats.
https://docs.biobricks.ai
MIT License
5 stars 2 forks source link

Regression with biobricks-0.2.0: file not found error for missing directory `data/` #15

Closed zmughal closed 7 months ago

zmughal commented 7 months ago

Introduced in commit https://github.com/biobricks-ai/biobricks/commit/3d039fcdeb8087544a3f63d58040909776f0cf88. Seems that find_allowed_files() does not check for the existence of dir_path first.

Also, is it only supposed to look in data/ (seems that searching brick/ got dropped)?

$ biobricks install ice
2023-11-26 01:55:48 | INFO: getting latest version of https://github.com/biobricks-ai/ice
2023-11-26 01:55:49 | INFO: running checks on brick
2023-11-26 01:55:49 | INFO: git clone https://github.com/biobricks-ai/ice biobricks-ai/ice/f247c13d1e93feb5e8d7b46a4036f4485a8b5a43 in /mnt/ssd/biobricks
2023-11-26 01:55:49 | INFO: adding brick to dvc cache
2023-11-26 01:55:50 | INFO: setting up credentials for dvc.biobricks.ai
2023-11-26 01:55:51 | INFO: discovering brick assets dvc.biobricks.ai
Traceback (most recent call last):
  File "/path/to/.local/bin/biobricks", line 8, in <module>
    sys.exit(cli())
  File "/path/to/.local/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/path/to/.local/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/path/to/.local/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/path/to/.local/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/path/to/.local/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/path/to/.local/lib/python3.10/site-packages/biobricks/cli.py", line 71, in install
    return Brick.Resolve(ref, force_remote=True).install()
  File "/path/to/.local/lib/python3.10/site-packages/biobricks/brick.py", line 159, in install
    allowed_paths = find_allowed_files(fs, "data", Brick.ALLOWED_FILETYPES)
  File "/path/to/.local/lib/python3.10/site-packages/biobricks/brick.py", line 151, in find_allowed_files
    for file_path in fs.listdir(dir_path):
  File "/path/to/.local/lib/python3.10/site-packages/fsspec/spec.py", line 1514, in listdir
    return self.ls(path, detail=detail, **kwargs)
  File "/path/to/.local/lib/python3.10/site-packages/dvc/fs/dvc.py", line 336, in ls
    raise FileNotFoundError(errno.ENOENT, os.strerror(errno.ENOENT), path)
FileNotFoundError: [Errno 2] No such file or directory: 'data'

Work around: downgrade biobricks:

pip3 install biobricks==0.1.66
tomlue commented 7 months ago

fixed in 0.2.2. We really ought to normalize on using the brick dir though.