search

datalad / datalad-catalog

Create a user-friendly data catalog from structured metadata
https://datalad-catalog.netlify.app
MIT License
15 stars 12 forks source link

`next`'s version determines success/failure of workflow tests #335

Closed jsheunis closed 3 months ago

jsheunis commented 1 year ago

Running tests locally (on a mac) with current main https://github.com/datalad/datalad-catalog/commit/673281d89c2e39c3aff05d35a1ca33a5544dc9ee, and changing dependency version of datalad-next in setup.cfg:

With dependency datalad-next @ git+https://github.com/datalad/datalad-next.git@main all tests succeed locally, including datalad_catalog/tests/test_workflow.py::test_workflow_new.

Running this test with dependency datalad-next == 1.0.0b3 leads to a failure:

python -m pytest -s -vv datalad_catalog/tests/test_workflow.py::test_workflow_new
Output: ``` collected 1 item datalad_catalog/tests/test_workflow.py::test_workflow_new create(ok): . (dataset) [INFO ] Start Total [INFO ] Total 1 datasets done in 0.32008 sec at 3.12422 datasets/sec [INFO ] Start Total [INFO ] Total 1 datasets done in 0.795881 sec at 1.25647 datasets/sec [INFO ] Ensuring presence of Dataset(/private/var/folders/g7/720mm8ns7hg3d28_3yyzkx_r0000gp/T/pytest-of-jsheunis/pytest-11/test_workflow_new0/workflow_superdataset) to get /private/var/folders/g7/720mm8ns7hg3d28_3yyzkx_r0000gp/T/pytest-of-jsheunis/pytest-11/test_workflow_new0/workflow_superdataset [INFO ] Start Installing [INFO ] Installing 1 datasets done in 0.0378249 sec at 26.4376 datasets/sec [INFO ] Start Core metadata extraction [INFO ] Core metadata extraction 2 Files done in 0.158409 sec at 12.6256 Files/sec [INFO ] Start Studyminimeta metadata extraction [INFO ] Studyminimeta metadata extraction 2 Files done in 0.0099268 sec at 201.475 Files/sec catalog_add(ok): /private/var/folders/g7/720mm8ns7hg3d28_3yyzkx_r0000gp/T/pytest-of-jsheunis/pytest-11/test_workflow_new0/workflow_catalog [Metadata record successfully added to catalog (dataset: dataset_id=f6b68942-09c9-4abb-976d-990e73c10d05, dataset_version=a6cb304aca5c75ee92eee86584bf389cf128cc53)] catalog_add(ok): /private/var/folders/g7/720mm8ns7hg3d28_3yyzkx_r0000gp/T/pytest-of-jsheunis/pytest-11/test_workflow_new0/workflow_catalog [Metadata record successfully updated in catalog (dataset: dataset_id=f6b68942-09c9-4abb-976d-990e73c10d05, dataset_version=a6cb304aca5c75ee92eee86584bf389cf128cc53)] action summary: catalog_add (ok: 2) foreach-dataset(ok): /private/var/folders/g7/720mm8ns7hg3d28_3yyzkx_r0000gp/T/pytest-of-jsheunis/pytest-11/test_workflow_new0/workflow_superdataset (dataset) [INFO ] Start Core metadata extraction [INFO ] Core metadata extraction 1 Files done in 0.109497 sec at 9.13268 Files/sec [WARNING] MetadataRecord extractor datacite_gin from distribution datalad-catalog overrides metadata extractor from distribution datalad-next foreach-dataset(error): /private/var/folders/g7/720mm8ns7hg3d28_3yyzkx_r0000gp/T/pytest-of-jsheunis/pytest-11/test_workflow_new0/workflow_superdataset/some_dir/subdataset (dataset) [(datalad-next 0.2.0+4.gc463648.dirty (/Users/jsheunis/Documents/psyinf/datalad-catalog), Requirement.parse('datalad-next>=1.0.0b3'))] FAILEDVersions: annexremote=1.6.0 boto=2.49.0 cmd:7z=17.04 cmd:annex=10.20230227 cmd:bundled-git=UNKNOWN cmd:git=2.31.1 cmd:ssh=8.1p1 cmd:system-git=2.31.1 cmd:system-ssh=8.1p1 datalad=0.19.2 humanize=4.4.0 iso8601=1.1.0 keyring=23.13.1 keyrings.alt=UNKNOWN msgpack=1.0.4 platformdirs=2.6.2 requests=2.28.2 Obscure filename: str=b' |;&%b5{}\'"<>\xce\x94\xd7\xa7\xd9\x85\xe0\xb9\x97\xe3\x81\x82 .datc ' repr=' |;&%b5{}\'"<>Δקم๗あ .datc ' Encodings: default='utf-8' filesystem='utf-8' locale.prefered='UTF-8' Environment: LC_TERMINAL_VERSION='3.4.5' LC_CTYPE='en_US.UTF-8' PATH='/Users/jsheunis/.rbenv/shims:/Users/jsheunis/opt/miniconda3/envs/meow3/bin:/Users/jsheunis/opt/miniconda3/condabin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Library/Apple/usr/bin:/Users/jsheunis/.dotfiles/bin:/usr/local/sbin:/usr/X11R6/bin:/usr/local/games:/usr/games' LC_TERMINAL='iTerm2' LANG='en_US.UTF-8' LANGUAGE='en_US.UTF-8' LC_ALL='en_US.UTF-8' GIT_CONFIG_PARAMETERS="'init.defaultBranch=dl-test-branch' 'clone.defaultRemoteName=dl-test-remote'" PYTHON_KEYRING_BACKEND='keyrings.alt.file.PlaintextKeyring' GIT_ASKPASS='true' ================================================= FAILURES ================================================== _____________________________________________ test_workflow_new _____________________________________________ test_data = workflow_catalog_path = PosixPath('/private/var/folders/g7/720mm8ns7hg3d28_3yyzkx_r0000gp/T/pytest-of-jsheunis/pytest-11/test_workflow_new0/workflow_catalog') workflow_dataset_path = PosixPath('/private/var/folders/g7/720mm8ns7hg3d28_3yyzkx_r0000gp/T/pytest-of-jsheunis/pytest-11/test_workflow_new0/workflow_superdataset') @pytest.mark.skipif( sys.platform == "win32", reason="jq does not build on windows" ) @skip_if_adjusted_branch def test_workflow_new(test_data, workflow_catalog_path, workflow_dataset_path): cat_path = workflow_catalog_path super_path = workflow_dataset_path ckwa = dict(result_renderer="disabled") # Create super and subdataset, save all sub_ds = create(super_path / "some_dir" / "subdataset", force=True, **ckwa) sub_ds.save(to_git=True, **ckwa) super_ds = create(super_path, force=True, **ckwa) super_ds.save(to_git=True, **ckwa) assert_repo_status(super_ds.path) # Test if metadata files exist assert (Path(super_ds.path) / ".studyminimeta.yaml").exists() assert (Path(sub_ds.path) / "datacite.yml").exists() # Create catalog cat_path = Path(cat_path) cat = WebCatalog(location=cat_path) cat.create(config_file=test_data.workflow_config_file, force=True) assert cat_path.exists() assert cat_path.is_dir() for p in catalog_paths: pth = cat_path / p assert pth.exists() # Run workflow extractors = [ "metalad_core", "metalad_studyminimeta", "datacite_gin", ] tuple(super_workflow(super_ds, cat, extractors)) # TODO: test interim workflow outputs, including: # - extracted metadata file (amount of lines?) # - translated metadata file (amount of lines?) assert (Path(super_ds.path) / "extracted_meta.json").exists() assert (Path(sub_ds.path) / "extracted_meta.json").exists() assert (Path(super_ds.path) / "translated_meta.json").exists() > assert (Path(sub_ds.path) / "translated_meta.json").exists() E AssertionError: assert False E + where False = () E + where = (PosixPath('/private/var/folders/g7/720mm8ns7hg3d28_3yyzkx_r0000gp/T/pytest-of-jsheunis/pytest-11/test_workflow_new0/workflow_superdataset/some_dir/subdataset') / 'translated_meta.json').exists E + where PosixPath('/private/var/folders/g7/720mm8ns7hg3d28_3yyzkx_r0000gp/T/pytest-of-jsheunis/pytest-11/test_workflow_new0/workflow_superdataset/some_dir/subdataset') = Path('/private/var/folders/g7/720mm8ns7hg3d28_3yyzkx_r0000gp/T/pytest-of-jsheunis/pytest-11/test_workflow_new0/workflow_superdataset/some_dir/subdataset') E + where '/private/var/folders/g7/720mm8ns7hg3d28_3yyzkx_r0000gp/T/pytest-of-jsheunis/pytest-11/test_workflow_new0/workflow_superdataset/some_dir/subdataset' = Dataset('/private/var/folders/g7/720mm8ns7hg3d28_3yyzkx_r0000gp/T/pytest-of-jsheunis/pytest-11/test_workflow_new0/workflow_superdataset/some_dir/subdataset').path datalad_catalog/tests/test_workflow.py:82: AssertionError --------------------------------------------- Captured log call --------------------------------------------- INFO datalad:log.py:432 Clear progress bars INFO datalad:log.py:432 Refresh progress bars INFO datalad:progressbars.py:131 Start Total INFO datalad.core.local.save:log.py:432 Total: starting INFO datalad.core.local.save:log.py:432 INFO datalad.core.local.save:log.py:432 Total: processed result for /private/var/folders/g7/720mm8ns7hg3d28_3yyzkx_r0000gp/T/pytest-of-jsheunis/pytest-11/test_workflow_new0/workflow_superdataset/some_dir/subdataset INFO datalad:progressbars.py:180 Total 1 datasets done in 0.32008 sec at 3.12422 datasets/sec INFO datalad.core.local.save:log.py:432 Total: done INFO datalad:progressbars.py:131 Start Total INFO datalad.core.local.save:log.py:432 Total: starting INFO datalad.core.local.save:log.py:432 INFO datalad.core.local.save:log.py:432 Total: processed result for /private/var/folders/g7/720mm8ns7hg3d28_3yyzkx_r0000gp/T/pytest-of-jsheunis/pytest-11/test_workflow_new0/workflow_superdataset INFO datalad:progressbars.py:180 Total 1 datasets done in 0.795881 sec at 1.25647 datasets/sec INFO datalad.core.local.save:log.py:432 Total: done INFO datalad.distribution.get:get.py:626 Ensuring presence of Dataset(/private/var/folders/g7/720mm8ns7hg3d28_3yyzkx_r0000gp/T/pytest-of-jsheunis/pytest-11/test_workflow_new0/workflow_superdataset) to get /private/var/folders/g7/720mm8ns7hg3d28_3yyzkx_r0000gp/T/pytest-of-jsheunis/pytest-11/test_workflow_new0/workflow_superdataset INFO datalad:progressbars.py:131 Start Installing INFO datalad.distribution.get:log.py:432 Installing: starting INFO datalad.distribution.get:log.py:432 INFO datalad.distribution.get:log.py:432 Installing: processed result for /private/var/folders/g7/720mm8ns7hg3d28_3yyzkx_r0000gp/T/pytest-of-jsheunis/pytest-11/test_workflow_new0/workflow_superdataset/some_dir/subdataset INFO datalad:progressbars.py:180 Installing 1 datasets done in 0.0378249 sec at 26.4376 datasets/sec INFO datalad.distribution.get:log.py:432 Installing: done INFO datalad:progressbars.py:131 Start Core metadata extraction INFO datalad.metadata.extractors.metalad_core:log.py:432 Start core metadata extraction from Dataset(/private/var/folders/g7/720mm8ns7hg3d28_3yyzkx_r0000gp/T/pytest-of-jsheunis/pytest-11/test_workflow_new0/workflow_superdataset) INFO datalad.metadata.extractors.metalad_core:log.py:432 Extracted core metadata from /private/var/folders/g7/720mm8ns7hg3d28_3yyzkx_r0000gp/T/pytest-of-jsheunis/pytest-11/test_workflow_new0/workflow_superdataset INFO datalad:progressbars.py:180 Core metadata extraction 2 Files done in 0.158409 sec at 12.6256 Files/sec INFO datalad.metadata.extractors.metalad_core:log.py:432 Finished core metadata extraction from Dataset(/private/var/folders/g7/720mm8ns7hg3d28_3yyzkx_r0000gp/T/pytest-of-jsheunis/pytest-11/test_workflow_new0/workflow_superdataset) INFO datalad:progressbars.py:131 Start Studyminimeta metadata extraction INFO datalad.metadata.extractors.studyminimeta:log.py:432 Start studyminimeta metadata extraction from /private/var/folders/g7/720mm8ns7hg3d28_3yyzkx_r0000gp/T/pytest-of-jsheunis/pytest-11/test_workflow_new0/workflow_superdataset INFO datalad:progressbars.py:180 Studyminimeta metadata extraction 2 Files done in 0.0099268 sec at 201.475 Files/sec INFO datalad.metadata.extractors.studyminimeta:log.py:432 Finished studyminimeta metadata extraction from /private/var/folders/g7/720mm8ns7hg3d28_3yyzkx_r0000gp/T/pytest-of-jsheunis/pytest-11/test_workflow_new0/workflow_superdataset INFO datalad:log.py:432 Clear progress bars INFO datalad:log.py:432 Refresh progress bars INFO datalad:log.py:432 Clear progress bars INFO datalad:log.py:432 Refresh progress bars INFO datalad:log.py:432 Clear progress bars INFO datalad:log.py:432 Refresh progress bars INFO datalad:log.py:432 Clear progress bars INFO datalad:log.py:432 Refresh progress bars INFO datalad:progressbars.py:131 Start Core metadata extraction INFO datalad.metadata.extractors.metalad_core:log.py:432 Start core metadata extraction from Dataset(/private/var/folders/g7/720mm8ns7hg3d28_3yyzkx_r0000gp/T/pytest-of-jsheunis/pytest-11/test_workflow_new0/workflow_superdataset/some_dir/subdataset) INFO datalad.metadata.extractors.metalad_core:log.py:432 Extracted core metadata from /private/var/folders/g7/720mm8ns7hg3d28_3yyzkx_r0000gp/T/pytest-of-jsheunis/pytest-11/test_workflow_new0/workflow_superdataset/some_dir/subdataset INFO datalad:progressbars.py:180 Core metadata extraction 1 Files done in 0.109497 sec at 9.13268 Files/sec INFO datalad.metadata.extractors.metalad_core:log.py:432 Finished core metadata extraction from Dataset(/private/var/folders/g7/720mm8ns7hg3d28_3yyzkx_r0000gp/T/pytest-of-jsheunis/pytest-11/test_workflow_new0/workflow_superdataset/some_dir/subdataset) WARNING datalad.metadata.extract:extract.py:523 MetadataRecord extractor datacite_gin from distribution datalad-catalog overrides metadata extractor from distribution datalad-next INFO datalad:log.py:432 Clear progress bars INFO datalad:log.py:432 Refresh progress bars ============================================= warnings summary ============================================== ../../../opt/miniconda3/envs/meow3/lib/python3.10/site-packages/boto/plugin.py:40 /Users/jsheunis/opt/miniconda3/envs/meow3/lib/python3.10/site-packages/boto/plugin.py:40: DeprecationWarning: the imp module is deprecated in favour of importlib and slated for removal in Python 3.12; see the module's documentation for alternative uses import imp ../../../opt/miniconda3/envs/meow3/lib/python3.10/site-packages/datalad/interface/utils.py:223 ../../../opt/miniconda3/envs/meow3/lib/python3.10/site-packages/datalad/interface/utils.py:223 ../../../opt/miniconda3/envs/meow3/lib/python3.10/site-packages/datalad/interface/utils.py:223 ../../../opt/miniconda3/envs/meow3/lib/python3.10/site-packages/datalad/interface/utils.py:223 ../../../opt/miniconda3/envs/meow3/lib/python3.10/site-packages/datalad/interface/utils.py:223 ../../../opt/miniconda3/envs/meow3/lib/python3.10/site-packages/datalad/interface/utils.py:223 ../../../opt/miniconda3/envs/meow3/lib/python3.10/site-packages/datalad/interface/utils.py:223 /Users/jsheunis/opt/miniconda3/envs/meow3/lib/python3.10/site-packages/datalad/interface/utils.py:223: DeprecationWarning: datalad.interface.utils.eval_results is obsolete. Use datalad.interface.base.eval_results instead warnings.warn("datalad.interface.utils.eval_results is obsolete. " ../../../opt/miniconda3/envs/meow3/lib/python3.10/site-packages/datalad/tests/utils_pytest.py:74 /Users/jsheunis/opt/miniconda3/envs/meow3/lib/python3.10/site-packages/datalad/tests/utils_pytest.py:74: PytestUnknownMarkWarning: Unknown pytest.mark.skip_if_adjusted_branch - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html return getattr(pytest.mark, name) -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ========================================== short test summary info ========================================== FAILED datalad_catalog/tests/test_workflow.py::test_workflow_new - AssertionError: assert False ```

The failure involves testing for the existence of a file at a path, which was generated during the test.

jsheunis commented 1 year ago

And funnily enough, this test succeeds on appveyor:

https://ci.appveyor.com/project/mih/datalad-catalog/builds/47949235/job/akml23rhm4a3a87o#L1921

(https://github.com/datalad/datalad-catalog/commit/983a429829c6dffedba53b61e141c8bc0b48af24)

jsheunis commented 3 months ago

IIRC this was an issue because of a clash between my virtual environment and my Mac's base environment. I.e. originated and solved locally.