datalad / datalad-deprecated

DataLad extension for functionality that has been phased out of the core package
Other
0 stars 3 forks source link

RF: migrate from using nose for testing to pytest #51

Closed yarikoptic closed 1 year ago

yarikoptic commented 2 years ago

Closes #66

yarikoptic commented 2 years ago

something is odd with a test on crippled FS (FWIW: uses python 3.8) -- needs "looking into".

yarikoptic commented 2 years ago
fails here: 18 failed, 29 passed, 17 skipped, 89 warnings in 238.48s -- 64 total ```shell =========================== short test summary info ============================ FAILED ../tests/test_annotate_paths.py::test_get_modified_subpaths - Assertio... FAILED ../tests/test_auto.py::test_proxying_open_testrepobased - Failed: DID ... FAILED ../tests/test_auto.py::test_proxying_open_h5py - AssertionError FAILED ../tests/test_auto.py::test_proxying_open_regular - AssertionError FAILED ../tests/test_auto.py::test_proxying_io_open_regular - AssertionError FAILED ../tests/test_auto.py::test_proxying_lzma_LZMAFile - _lzma.LZMAError: ... FAILED ../tests/test_auto.py::test_proxying_open_nibabel - nibabel.filebasedi... FAILED ../tests/test_auto.py::test_proxying_os_stat - AssertionError: assert ... FAILED ../tests/test_create_sibling_webui.py::test_preserve_attrs - assert 12... FAILED ../tests/test_diff.py::test_diff - AssertionError: Got 0 instead of 1 ... FAILED ../tests/test_diff.py::test_diff_recursive - AssertionError: Got 0 ins... FAILED ../tests/test_diff.py::test_diff_helper - AssertionError: Got 4 instea... FAILED ../tests/test_ls_webui.py::test_ls_json - AttributeError: 'NoneType' o... FAILED ../tests/test_publish.py::test_since_empty_and_unsupported - datalad.s... FAILED ../tests/test_publish.py::test_publish_simple - AssertionError: Got 0 ... FAILED ../tests/test_publish.py::test_publish_recursive - datalad.support.exc... FAILED ../tests/test_publish.py::test_publish_with_data - datalad.support.exc... FAILED ../tests/test_publish.py::test_gh1426 - datalad.support.exceptions.Inc... ====== 18 failed, 29 passed, 17 skipped, 89 warnings in 238.48s (0:03:58) ====== ```

and in a fresh PR #52 to see how master is doing with nose:

Ran 71 tests in 263.468s FAILED (SKIP=17, errors=12, failures=12) -- 30 pass if algebra is right ```shell datalad@smaug:/mnt/datasets/datalad/ci/datalad-deprecated/2022/04/28/pr/52/30e7257/github-crippled-filesystems-51-failed$ grep -e '\(ERROR\|FAIL\):' test/8_Run\ tests.txt | nl 1 2022-04-28T15:49:43.5995618Z ERROR: datalad_deprecated.tests.test_auto.test_proxying_lzma_LZMAFile(.generate_dat at 0x7fbebc05add0>, .verify_dat at 0x7fbebc030440>) 2 2022-04-28T15:49:43.6124495Z ERROR: datalad_deprecated.tests.test_auto.test_proxying_open_nibabel(.generate_nii at 0x7fbea2d2dd40>, .verify_nii at 0x7fbebc04d680>) 3 2022-04-28T15:49:43.6133455Z ERROR: datalad_deprecated.tests.test_create_sibling_webui.test_target_ssh_recursive(False,) 4 2022-04-28T15:49:43.6148023Z ERROR: datalad_deprecated.tests.test_create_sibling_webui.test_target_ssh_since(False,) 5 2022-04-28T15:49:43.6154795Z ERROR: datalad_deprecated.tests.test_create_sibling_webui.test_replace_and_relative_sshpath(False,) 6 2022-04-28T15:49:43.6199232Z ERROR: datalad_deprecated.tests.test_create_sibling_webui.test_target_ssh_inherit('backup', False, False) 7 2022-04-28T15:49:43.6213651Z ERROR: datalad_deprecated.tests.test_ls_webui.test_ls_json 8 2022-04-28T15:49:43.6220148Z ERROR: datalad_deprecated.tests.test_publish.test_target_ssh_inherit('backup', False, False) 9 2022-04-28T15:49:43.6239371Z ERROR: datalad_deprecated.tests.test_publish.test_gh1426 10 2022-04-28T15:49:43.6251938Z ERROR: datalad_deprecated.tests.test_publish.test_since_empty_and_unsupported 11 2022-04-28T15:49:43.6264670Z ERROR: datalad_deprecated.tests.test_publish.test_publish_recursive 12 2022-04-28T15:49:43.6347100Z ERROR: datalad_deprecated.tests.test_publish.test_publish_with_data 13 2022-04-28T15:49:43.6361088Z FAIL: datalad_deprecated.tests.test_annotate_paths.test_get_modified_subpaths 14 2022-04-28T15:49:43.6368538Z FAIL: datalad_deprecated.tests.test_auto.test_proxying_open_h5py(.generate_hdf5 at 0x7fbebbe10b00>, .verify_hdf5 at 0x7fbebbe104d0>) 15 2022-04-28T15:49:43.6373209Z FAIL: datalad_deprecated.tests.test_auto.test_proxying_open_regular(.generate_dat at 0x7fbebc073cb0>, .verify_dat at 0x7fbebc0738c0>) 16 2022-04-28T15:49:43.6378341Z FAIL: datalad_deprecated.tests.test_auto.test_proxying_io_open_regular(.generate_dat at 0x7fbebc016680>, .verify_dat at 0x7fbebbe10320>) 17 2022-04-28T15:49:43.6383583Z FAIL: datalad_deprecated.tests.test_auto.test_proxying_os_stat(.generate_dat at 0x7fbebbc727a0>, .verify_dat at 0x7fbebbc72ef0>) 18 2022-04-28T15:49:43.6388056Z FAIL: datalad_deprecated.tests.test_auto.test_proxying_open_testrepobased 19 2022-04-28T15:49:43.6392518Z FAIL: datalad_deprecated.tests.test_create_sibling_webui.test_failon_no_permissions(False,) 20 2022-04-28T15:49:43.6399817Z FAIL: datalad_deprecated.tests.test_create_sibling_webui.test_preserve_attrs 21 2022-04-28T15:49:43.6405885Z FAIL: datalad_deprecated.tests.test_diff.test_diff_helper 22 2022-04-28T15:49:43.6422432Z FAIL: datalad_deprecated.tests.test_diff.test_diff 23 2022-04-28T15:49:43.6429193Z FAIL: datalad_deprecated.tests.test_diff.test_diff_recursive 24 2022-04-28T15:49:43.6436482Z FAIL: datalad_deprecated.tests.test_publish.test_publish_simple ```
yarikoptic commented 1 year ago

windows and crippled and even ubuntu seems to error out with

collected 98 items / 1 error / 4 skipped                                                                                                                                                                                                       
==================================================================================================================== ERRORS ====================================================================================================================
_________________________________________________________________________________________________ ERROR collecting metadata/extractors/xmp.py __________________________________________________________________________________________________
../../../dlvenv/lib/python3.8/site-packages/datalad_deprecated/metadata/extractors/xmp.py:20: in <module>
    from libxmp.utils import file_to_dict
../../../dlvenv/lib/python3.8/site-packages/libxmp/__init__.py:50: in <module>
    from .core import XMPMeta, XMPIterator
../../../dlvenv/lib/python3.8/site-packages/libxmp/core.py:50: in <module>
    from . import exempi as _cexempi
../../../dlvenv/lib/python3.8/site-packages/libxmp/exempi.py:69: in <module>
    EXEMPI = _load_exempi()
../../../dlvenv/lib/python3.8/site-packages/libxmp/exempi.py:60: in _load_exempi
    raise ExempiLoadError('Exempi library not found.')
E   libxmp.ExempiLoadError: Exempi library not found.

looking into it

yarikoptic commented 1 year ago

many of "crippled" tests fail due to what looks like broken packaging of the datalad-deprecated as not shipping the test data

2022-09-29T14:26:19.5528040Z         if not follow_symlinks and os.path.islink(src):
2022-09-29T14:26:19.5528328Z             os.symlink(os.readlink(src), dst)
2022-09-29T14:26:19.5528555Z         else:
2022-09-29T14:26:19.5528825Z >           with open(src, 'rb') as fsrc:
2022-09-29T14:26:19.5529453Z E           FileNotFoundError: [Errno 2] No such file or directory: '/opt/hostedtoolcache/Python/3.7.14/x64/lib/python3.7/site-packages/datalad_deprecated/metadata/tests/data/exif.jpg'
2022-09-29T14:26:19.5529771Z 
2022-09-29T14:26:19.5529960Z /opt/hostedtoolcache/Python/3.7.14/x64/lib/python3.7/shutil.py:120: FileNotFoundError
2022-09-29T14:26:19.5530575Z ------------------------------ Captured log call -------------------------------

and in crippled we install via python -m pip install . so non-devel mode while in appveyor we do both

  - python -m pip install -r requirements-devel.txt
  - python -m pip install .

and we have

$> head requirements-devel.txt 
# requirements for a development environment
-e .[devel]

so I guess we do test "in place" installation and thus finding the data files...

edit0: above observation is not the answer since it seems we do the same double pip install in both and also do testing from __testhome__ in both. so it is smth else but still smells with the bug in "packaging".

edit1: for now I will postpone troubleshooting of crippled and concentrate on making appveyor healthy again. For crippled, which would entail also fixing real issues which were introduced recently -- work better be done AFTER this PR is merged so we have consistent testing base.

yarikoptic commented 1 year ago

ha -- we do get the same problems now on appveyor ubuntu since we do test those metadata tests now. So clearly packaging needs to be fixed up... looking into it.

codecov-commenter commented 1 year ago

Codecov Report

Base: 57.24% // Head: 80.37% // Increases project coverage by +23.12% :tada:

Coverage data is based on head (ad17b01) compared to base (0ca142e). Patch coverage: 90.32% of modified lines in pull request are covered.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #51 +/- ## =========================================== + Coverage 57.24% 80.37% +23.12% =========================================== Files 57 58 +1 Lines 5380 5351 -29 =========================================== + Hits 3080 4301 +1221 + Misses 2300 1050 -1250 ``` | [Impacted Files](https://codecov.io/gh/datalad/datalad-deprecated/pull/51?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=datalad) | Coverage Δ | | |---|---|---| | [datalad\_deprecated/\_\_init\_\_.py](https://codecov.io/gh/datalad/datalad-deprecated/pull/51/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=datalad#diff-ZGF0YWxhZF9kZXByZWNhdGVkL19faW5pdF9fLnB5) | `100.00% <ø> (ø)` | | | [datalad\_deprecated/metadata/\_\_init\_\_.py](https://codecov.io/gh/datalad/datalad-deprecated/pull/51/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=datalad#diff-ZGF0YWxhZF9kZXByZWNhdGVkL21ldGFkYXRhL19faW5pdF9fLnB5) | `100.00% <ø> (ø)` | | | [...deprecated/metadata/extractors/tests/test\_audio.py](https://codecov.io/gh/datalad/datalad-deprecated/pull/51/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=datalad#diff-ZGF0YWxhZF9kZXByZWNhdGVkL21ldGFkYXRhL2V4dHJhY3RvcnMvdGVzdHMvdGVzdF9hdWRpby5weQ==) | `93.93% <ø> (+53.93%)` | :arrow_up: | | [...ted/metadata/extractors/tests/test\_datacite\_xml.py](https://codecov.io/gh/datalad/datalad-deprecated/pull/51/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=datalad#diff-ZGF0YWxhZF9kZXByZWNhdGVkL21ldGFkYXRhL2V4dHJhY3RvcnMvdGVzdHMvdGVzdF9kYXRhY2l0ZV94bWwucHk=) | `100.00% <ø> (ø)` | | | [...\_deprecated/metadata/extractors/tests/test\_exif.py](https://codecov.io/gh/datalad/datalad-deprecated/pull/51/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=datalad#diff-ZGF0YWxhZF9kZXByZWNhdGVkL21ldGFkYXRhL2V4dHJhY3RvcnMvdGVzdHMvdGVzdF9leGlmLnB5) | `92.59% <ø> (+44.31%)` | :arrow_up: | | [.../extractors/tests/test\_frictionless\_datapackage.py](https://codecov.io/gh/datalad/datalad-deprecated/pull/51/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=datalad#diff-ZGF0YWxhZF9kZXByZWNhdGVkL21ldGFkYXRhL2V4dHJhY3RvcnMvdGVzdHMvdGVzdF9mcmljdGlvbmxlc3NfZGF0YXBhY2thZ2UucHk=) | `100.00% <ø> (+30.76%)` | :arrow_up: | | [...deprecated/metadata/extractors/tests/test\_image.py](https://codecov.io/gh/datalad/datalad-deprecated/pull/51/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=datalad#diff-ZGF0YWxhZF9kZXByZWNhdGVkL21ldGFkYXRhL2V4dHJhY3RvcnMvdGVzdHMvdGVzdF9pbWFnZS5weQ==) | `92.59% <ø> (+44.31%)` | :arrow_up: | | [...eprecated/metadata/extractors/tests/test\_rfc822.py](https://codecov.io/gh/datalad/datalad-deprecated/pull/51/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=datalad#diff-ZGF0YWxhZF9kZXByZWNhdGVkL21ldGFkYXRhL2V4dHJhY3RvcnMvdGVzdHMvdGVzdF9yZmM4MjIucHk=) | `100.00% <ø> (+30.76%)` | :arrow_up: | | [...d\_deprecated/metadata/extractors/tests/test\_xmp.py](https://codecov.io/gh/datalad/datalad-deprecated/pull/51/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=datalad#diff-ZGF0YWxhZF9kZXByZWNhdGVkL21ldGFkYXRhL2V4dHJhY3RvcnMvdGVzdHMvdGVzdF94bXAucHk=) | `100.00% <ø> (+75.86%)` | :arrow_up: | | [datalad\_deprecated/conftest.py](https://codecov.io/gh/datalad/datalad-deprecated/pull/51/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=datalad#diff-ZGF0YWxhZF9kZXByZWNhdGVkL2NvbmZ0ZXN0LnB5) | `16.66% <16.66%> (ø)` | | | ... and [47 more](https://codecov.io/gh/datalad/datalad-deprecated/pull/51/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=datalad) | | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=datalad). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=datalad)

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

yarikoptic commented 1 year ago

woohoo -- appveyor is green .

crippled still has 16 failing I will look into later/separately ... forgot to push some fixes -- will push soon ```shell FAILED ../tests/test_auto.py::test_proxying_open_testrepobased - Failed: DID ... FAILED ../tests/test_auto.py::test_proxying_open_h5py - AssertionError FAILED ../tests/test_auto.py::test_proxying_open_regular - AssertionError FAILED ../tests/test_auto.py::test_proxying_io_open_regular - AssertionError FAILED ../tests/test_auto.py::test_proxying_lzma_LZMAFile - _lzma.LZMAError: ... FAILED ../tests/test_auto.py::test_proxying_open_nibabel - nibabel.filebasedi... FAILED ../tests/test_auto.py::test_proxying_os_stat - AssertionError: assert ... FAILED ../tests/test_create_sibling_webui.py::test_target_ssh_recursive - dat... FAILED ../tests/test_create_sibling_webui.py::test_preserve_attrs - assert 12... FAILED ../tests/test_diff.py::test_diff_helper - AssertionError: Got 4 instea... FAILED ../tests/test_ls_webui.py::test_ls_json - AttributeError: 'NoneType' o... FAILED ../tests/test_publish.py::test_since_empty_and_unsupported - datalad.s... FAILED ../tests/test_publish.py::test_publish_simple - AssertionError: Got 0 ... FAILED ../tests/test_publish.py::test_publish_recursive - datalad.support.exc... FAILED ../tests/test_publish.py::test_publish_with_data - datalad.support.exc... FAILED ../tests/test_publish.py::test_gh1426 - datalad.support.exceptions.Inc... ```

also need to figure out why codecov is not reporting back, into statuses but it is known and shows above that we improve by 23% ;)

yarikoptic commented 1 year ago

ok -- it is very green. I will merge and release tomorrow unless any "stop" is voiced or feedback requiring changes is provided.

github-actions[bot] commented 1 year ago

:rocket: PR was released in 0.2.3 :rocket: