search

dandi / dandi-cli

DANDI command line client to facilitate common operations
https://dandi.readthedocs.io/
Apache License 2.0
21 stars 25 forks source link

`dandi/tests/test_files.py::test_validate_bogus - assert False` on `=dandi-cli-0.55.0` and `=dandi-cli-0.55.1` #1297

Closed TheChymera closed 1 year ago

TheChymera commented 1 year ago

One test fails for some reason. This is the full build log → https://ppb.chymera.eu/46db45.log

Any ideas what could be wrong?

jwodder commented 1 year ago

@TheChymera Cannot reproduce. If you print out [e.message for e in errors] after this line, what do you get?

TheChymera commented 1 year ago

@jwodder 

decohost /var/tmp/portage/dev-vcs/dandi-cli-0.55.1/work/dandi-0.55.1 # DANDI_TESTS_NONETWORK=1 pytest -vvs dandi/tests/test_files.py
=========================================================== test session starts ============================================================
platform linux -- Python 3.11.3, pytest-7.3.1, pluggy-1.0.0 -- /usr/bin/python3.11
cachedir: .pytest_cache
rootdir: /var/tmp/portage/dev-vcs/dandi-cli-0.55.1/work/dandi-0.55.1
configfile: tox.ini
plugins: mock-3.10.0, rerunfailures-11.1.2, pkgcore-0.12.21, pyfakefs-5.2.2
collected 14 items

dandi/tests/test_files.py::test_find_dandi_files PASSED
dandi/tests/test_files.py::test_find_dandi_files_with_bids PASSED
dandi/tests/test_files.py::test_dandi_file_zarr_with_excluded_dotfiles PASSED
dandi/tests/test_files.py::test_validate_simple1 PASSED
dandi/tests/test_files.py::test_validate_simple1_no_subject PASSED
dandi/tests/test_files.py::test_validate_simple2 PASSED
dandi/tests/test_files.py::test_validate_simple2_new PASSED
dandi/tests/test_files.py::test_validate_simple3_no_subject_id PASSED
dandi/tests/test_files.py::test_validate_bogus ['Unable to synchronously open file (file signature not found)']
FAILED
dandi/tests/test_files.py::test_upload_zarr SKIPPED (no network settings)
dandi/tests/test_files.py::test_zarr_properties PASSED
dandi/tests/test_files.py::test_upload_zarr_with_excluded_dotfiles SKIPPED (no network settings)
dandi/tests/test_files.py::test_validate_deep_zarr PASSED
dandi/tests/test_files.py::test_validate_zarr_deep_via_excluded_dotfiles PASSED

================================================================= FAILURES =================================================================
___________________________________________________________ test_validate_bogus ____________________________________________________________
dandi/tests/test_files.py:378: in test_validate_bogus
    assert any(e.message.startswith("Unable to open file") for e in errors)
E   assert False
E    +  where False = any(<generator object test_validate_bogus.<locals>.<genexpr> at 0x7f8d47d399a0>)
=========================================================== slowest 10 durations ===========================================================
2.01s setup    dandi/tests/test_files.py::test_validate_simple2
0.09s call     dandi/tests/test_files.py::test_validate_simple2
0.08s call     dandi/tests/test_files.py::test_validate_simple1
0.08s call     dandi/tests/test_files.py::test_validate_simple2_new
0.08s call     dandi/tests/test_files.py::test_validate_simple1_no_subject
0.07s call     dandi/tests/test_files.py::test_validate_simple3_no_subject_id
0.03s setup    dandi/tests/test_files.py::test_validate_simple3_no_subject_id
0.03s setup    dandi/tests/test_files.py::test_validate_simple1
0.02s call     dandi/tests/test_files.py::test_zarr_properties
0.01s call     dandi/tests/test_files.py::test_validate_bogus
========================================================= short test summary info ==========================================================
FAILED dandi/tests/test_files.py::test_validate_bogus - assert False
================================================= 1 failed, 11 passed, 2 skipped in 2.73s ==================================================

Can also confirm this continues to happen on dandi-0.55.1

jwodder commented 1 year ago

@TheChymera If you pass devel_debug=True to get_validation_errors(), what do you get?

TheChymera commented 1 year ago

@jwodder this is what I get:

decohost /var/tmp/portage/dev-vcs/dandi-cli-0.55.1/work/dandi-0.55.1 # DANDI_TESTS_NONETWORK=1 pytest -vvs dandi/tests/test_files.py
=========================================================== test session starts ============================================================
platform linux -- Python 3.11.3, pytest-7.3.1, pluggy-1.0.0 -- /usr/bin/python3.11
cachedir: .pytest_cache
rootdir: /var/tmp/portage/dev-vcs/dandi-cli-0.55.1/work/dandi-0.55.1
configfile: tox.ini
plugins: mock-3.10.0, rerunfailures-11.1.2, pkgcore-0.12.21, pyfakefs-5.2.2
collected 14 items

dandi/tests/test_files.py::test_find_dandi_files PASSED
dandi/tests/test_files.py::test_find_dandi_files_with_bids PASSED
dandi/tests/test_files.py::test_dandi_file_zarr_with_excluded_dotfiles PASSED
dandi/tests/test_files.py::test_validate_simple1 PASSED
dandi/tests/test_files.py::test_validate_simple1_no_subject PASSED
dandi/tests/test_files.py::test_validate_simple2 PASSED
dandi/tests/test_files.py::test_validate_simple2_new PASSED
dandi/tests/test_files.py::test_validate_simple3_no_subject_id PASSED
dandi/tests/test_files.py::test_validate_bogus FAILED
dandi/tests/test_files.py::test_upload_zarr SKIPPED (no network settings)
dandi/tests/test_files.py::test_zarr_properties PASSED
dandi/tests/test_files.py::test_upload_zarr_with_excluded_dotfiles SKIPPED (no network settings)
dandi/tests/test_files.py::test_validate_deep_zarr PASSED
dandi/tests/test_files.py::test_validate_zarr_deep_via_excluded_dotfiles PASSED

================================================================= FAILURES =================================================================
___________________________________________________________ test_validate_bogus ____________________________________________________________
dandi/tests/test_files.py:374: in test_validate_bogus
    errors = dandi_file(path).get_validation_errors(devel_debug=True)
dandi/files/bases.py:500: in get_validation_errors
    errors: list[ValidationResult] = pynwb_validate(
/usr/lib/python3.11/site-packages/fscacher/cache.py:142: in fingerprinter
    ret = f(*args, **kwargs)
dandi/pynwb_utils.py:362: in validate
    error_outputs, _ = pynwb.validate(paths=[path])
/usr/lib/python3.11/site-packages/hdmf/utils.py:649: in func_call
    return func(**pargs)
/usr/lib/python3.11/site-packages/pynwb/validate.py:143: in validate
    cached_namespaces, manager, namespace_dependencies = _get_cached_namespaces_to_validate(
/usr/lib/python3.11/site-packages/pynwb/validate.py:61: in _get_cached_namespaces_to_validate
    namespace_dependencies = NWBHDF5IO.load_namespaces(namespace_catalog=catalog, path=path, driver=driver)
/usr/lib/python3.11/site-packages/hdmf/utils.py:645: in func_call
    return func(args[0], **pargs)
/usr/lib/python3.11/site-packages/hdmf/backends/hdf5/h5tools.py:152: in load_namespaces
    open_file_obj = cls.__resolve_file_obj(path, file_obj, driver)
/usr/lib/python3.11/site-packages/hdmf/backends/hdf5/h5tools.py:127: in __resolve_file_obj
    file_obj = File(path, 'r', **file_kwargs)
/usr/lib/python3.11/site-packages/h5py/_hl/files.py:567: in __init__
    fid = make_fid(name, mode, userblock_size, fapl, fcpl, swmr=swmr)
/usr/lib/python3.11/site-packages/h5py/_hl/files.py:231: in make_fid
    fid = h5f.open(name, flags, fapl=fapl)
h5py/_objects.pyx:54: in h5py._objects.with_phil.wrapper
    ???
h5py/_objects.pyx:55: in h5py._objects.with_phil.wrapper
    ???
h5py/h5f.pyx:106: in h5py.h5f.open
    ???
E   OSError: Unable to synchronously open file (file signature not found)
=========================================================== slowest 10 durations ===========================================================
2.21s setup    dandi/tests/test_files.py::test_validate_simple2
0.15s call     dandi/tests/test_files.py::test_validate_simple1_no_subject
0.11s call     dandi/tests/test_files.py::test_validate_simple2_new
0.10s call     dandi/tests/test_files.py::test_validate_simple1
0.09s call     dandi/tests/test_files.py::test_validate_simple2
0.09s call     dandi/tests/test_files.py::test_validate_simple3_no_subject_id
0.03s setup    dandi/tests/test_files.py::test_validate_simple3_no_subject_id
0.03s setup    dandi/tests/test_files.py::test_validate_simple1
0.03s call     dandi/tests/test_files.py::test_zarr_properties
0.01s call     dandi/tests/test_files.py::test_validate_deep_zarr
========================================================= short test summary info ==========================================================
FAILED dandi/tests/test_files.py::test_validate_bogus - OSError: Unable to synchronously open file (file signature not found)
================================================= 1 failed, 11 passed, 2 skipped in 3.13s ==================================================
jwodder commented 1 year ago

@TheChymera How did you install h5py? If you did not use a prebuilt image (e.g., a wheel from PyPI), what version of hdf5 did you use?

TheChymera commented 1 year ago

@jwodder I have =hdf5-1.14.1 It's installed via my package manager based on the source tarball. I'm re-installing it to capture the build log and link it here. Anything in particular I should look for?

jwodder commented 1 year ago

@TheChymera Check that #1301 fixes the failure.

TheChymera commented 1 year ago

@jwodder here is the full build log in case that's still relevant → https://ppb.chymera.eu/3ca240.log

github-actions[bot] commented 1 year ago

:rocket: Issue was released in 0.56.0 :rocket: