Collecting tests fails due to git submodules and symlinks

[wholtz]

My issue is similar to #139 and I'm on the most recent commit on the development branch (ef7f312). The error appear to be related to copying a symlink within a git submodule:

$ git clone --recurse-submodules https://github.com/wholtz/pytest-workflow-submodule-issue.git
Cloning into 'pytest-workflow-submodule-issue'...
remote: Enumerating objects: 9, done.
remote: Counting objects: 100% (9/9), done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 9 (delta 0), reused 9 (delta 0), pack-reused 0
Receiving objects: 100% (9/9), done.
Submodule 'ci-functions' (https://code.jgi.doe.gov/user-support-analysis/ci-functions.git) registered for path '.ci-functions'
Cloning into '/Users/WJHoltz/pytest-workflow-submodule-issue/.ci-functions'...
remote: Enumerating objects: 608, done.        
remote: Counting objects: 100% (4/4), done.        
remote: Compressing objects: 100% (4/4), done.        
remote: Total 608 (delta 0), reused 0 (delta 0), pack-reused 604        
Receiving objects: 100% (608/608), 98.69 KiB | 5.80 MiB/s, done.
Resolving deltas: 100% (373/373), done.
Submodule path '.ci-functions': checked out '001b2feb1bcd641589571bb6e13a37e998b36057'
Submodule 'test/bats' (https://github.com/bats-core/bats-core.git) registered for path '.ci-functions/test/bats'
Submodule 'test/test_helper/bats-assert' (https://github.com/bats-core/bats-assert.git) registered for path '.ci-functions/test/test_helper/bats-assert'
Submodule 'test/test_helper/bats-support' (https://github.com/bats-core/bats-support.git) registered for path '.ci-functions/test/test_helper/bats-support'
Cloning into '/Users/WJHoltz/pytest-workflow-submodule-issue/.ci-functions/test/bats'...
remote: Enumerating objects: 7709, done.        
remote: Counting objects: 100% (560/560), done.        
remote: Compressing objects: 100% (149/149), done.        
remote: Total 7709 (delta 430), reused 494 (delta 390), pack-reused 7149        
Receiving objects: 100% (7709/7709), 1.87 MiB | 5.99 MiB/s, done.
Resolving deltas: 100% (4662/4662), done.
Cloning into '/Users/WJHoltz/pytest-workflow-submodule-issue/.ci-functions/test/test_helper/bats-assert'...
remote: Enumerating objects: 566, done.        
remote: Counting objects: 100% (48/48), done.        
remote: Compressing objects: 100% (23/23), done.        
remote: Total 566 (delta 35), reused 25 (delta 25), pack-reused 518        
Receiving objects: 100% (566/566), 145.47 KiB | 2.14 MiB/s, done.
Resolving deltas: 100% (347/347), done.
Cloning into '/Users/WJHoltz/pytest-workflow-submodule-issue/.ci-functions/test/test_helper/bats-support'...
remote: Enumerating objects: 121, done.        
remote: Counting objects: 100% (16/16), done.        
remote: Compressing objects: 100% (8/8), done.        
remote: Total 121 (delta 4), reused 13 (delta 4), pack-reused 105        
Receiving objects: 100% (121/121), 34.26 KiB | 3.43 MiB/s, done.
Resolving deltas: 100% (43/43), done.
Submodule path '.ci-functions/test/bats': checked out 'dc312135a8de97e149cd0307c45c1e41061c918d'
Submodule path '.ci-functions/test/test_helper/bats-assert': checked out '78fa631d1370562d2cd4a1390989e706158e7bf0'
Submodule path '.ci-functions/test/test_helper/bats-support': checked out '3c8fadc5097c9acfc96d836dced2bb598e48b009'
$ pytest --git-aware
============================================================================================================================== test session starts ===============================================================================================================================
platform darwin -- Python 3.11.0, pytest-7.2.0, pluggy-1.0.0
rootdir: /Users/WJHoltz/pytest-workflow-submodule-issue
plugins: workflow-1.7.0.dev0
collecting ... 
collected 0 items / 1 error                                                                                                                                                                                                                                                      

===================================================================================================================================== ERRORS =====================================================================================================================================
_____________________________________________________________________________________________________________________ ERROR collecting tests/test_dummy.yaml _____________________________________________________________________________________________________________________
../Applications/miniconda3/envs/wdl-dev/lib/python3.11/site-packages/pytest_workflow/plugin.py:445: in collect
    workflow = self.queue_workflow()
../Applications/miniconda3/envs/wdl-dev/lib/python3.11/site-packages/pytest_workflow/plugin.py:402: in queue_workflow
    duplicate_tree(root_dir, tempdir,
../Applications/miniconda3/envs/wdl-dev/lib/python3.11/site-packages/pytest_workflow/util.py:183: in duplicate_tree
    copy(src_path, dest_path)
../Applications/miniconda3/envs/wdl-dev/lib/python3.11/shutil.py:436: in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
../Applications/miniconda3/envs/wdl-dev/lib/python3.11/shutil.py:256: in copyfile
    with open(src, 'rb') as fsrc:
E   IsADirectoryError: [Errno 21] Is a directory: '/Users/WJHoltz/pytest-workflow-submodule-issue/.ci-functions/test/bats/test/fixtures/suite/recursive_with_symlinks/subsuite'
============================================================================================================================ short test summary info =============================================================================================================================
ERROR tests/test_dummy.yaml::dummy - IsADirectoryError: [Errno 21] Is a directory: '/Users/WJHoltz/pytest-workflow-submodule-issue/.ci-functions/test/bats/test/fixtures/suite/recursive_with_symlinks/subsuite'
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
================================================================================================================================ 1 error in 1.02s ================================================================================================================================
$ ls -l /Users/WJHoltz/pytest-workflow-submodule-issue/.ci-functions/test/bats/test/fixtures/suite/recursive_with_symlinks/subsuite
lrwxr-xr-x  1 WJHoltz  staff  22 Dec 20 11:11 /Users/WJHoltz/pytest-workflow-submodule-issue/.ci-functions/test/bats/test/fixtures/suite/recursive_with_symlinks/subsuite -> ../recursive/subsuite/
$ ls -l /Users/WJHoltz/pytest-workflow-submodule-issue/.ci-functions/test/bats/test/fixtures/suite/recursive/subsuite/
total 8
-rw-r--r--  1 WJHoltz  staff  40 Dec 20 11:11 test2.bats
$

Please let me know if I can provide any other useful information.

[rhpvorderman]

First of all, thank you for testing the latest development branch. That is really useful. Can you check the branch in https://github.com/LUMC/pytest-workflow/pull/164 to see if the issue still occurs? pip install git+https://github.com/lumc/pytest-workflow.git@issue162

[wholtz]

The fix is working for me. Thanks for the rapid changes!

[rhpvorderman]

Thanks for reporting back to us!