search

Amber-MD / pytraj

Python interface of cpptraj
https://amber-md.github.io/pytraj
168 stars 38 forks source link

Spurious Test Failures with Cpptraj #1633

Closed drroe closed 1 year ago

drroe commented 1 year ago

@hainm @swails

I'm going to start an issue to try to keep track of these strange failures. My local pytraj install (via anaconda 3 w/ python 3.8.12) does not have any of these.

https://jenkins.jasonswails.com/job/amber-github/job/pytraj/601/execution/node/22/log/

____________________ TestRandomizeIons.test_randomize_ions _____________________

self = <test_analysis.test_randomize_ions.TestRandomizeIons testMethod=test_randomize_ions>

    def test_randomize_ions(self):
        fn = os.path.join(cpptraj_test_dir, 'Test_RandomizeIons',
                          'adh206.tip3p.rst7.gz')
        tn = os.path.join(cpptraj_test_dir, 'Test_RandomizeIons',
                          'adh206.ff10.tip3p.parm7.gz')
        saved_traj_name = os.path.join(cpptraj_test_dir, 'Test_RandomizeIons',
                                       'around.overlap.rst7.save')
        # Set default RNG back to Marsaglia
        pt.set_default_rng(0)
        traj = pt.iterload(fn, tn)
        traj_mut = traj[:]
>       saved_traj = pt.iterload(saved_traj_name, traj.top)

test_analysis/test_randomize_ions.py:24: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/.local/lib/python3.7/site-packages/pytraj-2.0.6.dev0-py3.7-linux-x86_64.egg/pytraj/io.py:220: in iterload
    return load_traj(*args, **kwd)
/.local/lib/python3.7/site-packages/pytraj-2.0.6.dev0-py3.7-linux-x86_64.egg/pytraj/io.py:249: in load_traj
    ts._load(filename)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = pytraj.TrajectoryIterator, 0 frames: 
Size: 0.000000 (GB)
<Topology: 10849 atoms, 3469 residues, 3455 mols, PBC with box type = octahedral>

filename = '/cpptraj/test/Test_RandomizeIons/around.overlap.rst7.save'
top = None, frame_slice = (0, -1, 1), stride = None

    def _load(self,
              filename=None,
              top=None,
              frame_slice=(0, -1, 1),
              stride=None):
        """load trajectory/trajectories from filename/filenames
        with a single frame_slice or a list of frame_slice

        Notes
        -----
        if stride is not None, frame_slice will be ignored
        """
        if not top:
            top_ = self.top
        else:
            top_ = top

        frame_slice_ = frame_slice if stride is None else (0, -1, stride)

        if isinstance(filename, str) and os.path.exists(filename):
            super(TrajectoryIterator, self)._load(filename, top_, frame_slice_)
            self._frame_slice_list.append(frame_slice_)
        elif isinstance(filename,
                        str) and not os.path.exists(filename):

            flist = sort_filename_by_number(glob(filename))
            if not flist:
                if "\\" in filename:
                    # filename with white space
                    flist = [filename]
                if not flist:
                    raise ValueError(
>                       "Must provie a filename or list of filenames or file pattern"
                    )
E                   ValueError: Must provie a filename or list of filenames or file pattern

/.local/lib/python3.7/site-packages/pytraj-2.0.6.dev0-py3.7-linux-x86_64.egg/pytraj/trajectory/trajectory_iterator.py:218: ValueError
drroe commented 1 year ago

https://jenkins.jasonswails.com/blue/organizations/jenkins/amber-github%2Fpytraj/detail/pytraj/599/pipeline/

run_simple_test.py::TestRunnable::test_loading PASSED
run_simple_test.py::TestRunnable::test_other other stuff. throw all tests don't belong anywhere else here
PASSED
test_crdframes.py::TestCrdFrames::test_crdframes Fatal Python error: Segmentation fault

Current thread 0x00007fc3a8961740 (most recent call first):
  File "/.local/lib/python3.7/site-packages/pytraj-2.0.6.dev0-py3.7-linux-x86_64.egg/pytraj/analysis/rmsd.py", line 356 in rmsd
  File "/.local/lib/python3.7/site-packages/pytraj-2.0.6.dev0-py3.7-linux-x86_64.egg/pytraj/utils/decorators.py", line 11 in inner
  File "/home/jenkins/jenkins-cpu/workspace/amber-github/pytraj/tests/test_crdframes.py", line 27 in test_crdframes
  File "/opt/conda/lib/python3.7/unittest/case.py", line 628 in run
  File "/opt/conda/lib/python3.7/unittest/case.py", line 676 in __call__
  File "/opt/conda/lib/python3.7/site-packages/_pytest/unittest.py", line 333 in runtest
  File "/opt/conda/lib/python3.7/site-packages/_pytest/runner.py", line 169 in pytest_runtest_call
  File "/opt/conda/lib/python3.7/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/opt/conda/lib/python3.7/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/opt/conda/lib/python3.7/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/opt/conda/lib/python3.7/site-packages/_pytest/runner.py", line 262 in <lambda>
  File "/opt/conda/lib/python3.7/site-packages/_pytest/runner.py", line 341 in from_call
  File "/opt/conda/lib/python3.7/site-packages/_pytest/runner.py", line 262 in call_runtest_hook
  File "/opt/conda/lib/python3.7/site-packages/_pytest/runner.py", line 222 in call_and_report
  File "/opt/conda/lib/python3.7/site-packages/_pytest/runner.py", line 133 in runtestprotocol
  File "/opt/conda/lib/python3.7/site-packages/_pytest/runner.py", line 114 in pytest_runtest_protocol
  File "/opt/conda/lib/python3.7/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/opt/conda/lib/python3.7/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/opt/conda/lib/python3.7/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/opt/conda/lib/python3.7/site-packages/_pytest/main.py", line 348 in pytest_runtestloop
  File "/opt/conda/lib/python3.7/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/opt/conda/lib/python3.7/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/opt/conda/lib/python3.7/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/opt/conda/lib/python3.7/site-packages/_pytest/main.py", line 323 in _main
  File "/opt/conda/lib/python3.7/site-packages/_pytest/main.py", line 269 in wrap_session
  File "/opt/conda/lib/python3.7/site-packages/_pytest/main.py", line 316 in pytest_cmdline_main
  File "/opt/conda/lib/python3.7/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/opt/conda/lib/python3.7/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/opt/conda/lib/python3.7/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/opt/conda/lib/python3.7/site-packages/_pytest/config/__init__.py", line 167 in main
  File "/opt/conda/lib/python3.7/site-packages/_pytest/config/__init__.py", line 189 in console_main
  File "/opt/conda/bin/pytest", line 8 in <module>
Segmentation fault (core dumped)
hainm commented 1 year ago

Noted about this issue.

Hai

drroe commented 1 year ago

Another crdfames segfault. https://jenkins.jasonswails.com/blue/organizations/jenkins/amber-github%2Fpytraj/detail/pytraj/603/pipeline/

test_crdframes.py::TestCrdFrames::test_crdframes Fatal Python error: Segmentation fault

Current thread 0x00007f13b035a740 (most recent call first):
  File "/.local/lib/python3.7/site-packages/pytraj-2.0.6.dev0-py3.7-linux-x86_64.egg/pytraj/analysis/rmsd.py", line 356 in rmsd
  File "/.local/lib/python3.7/site-packages/pytraj-2.0.6.dev0-py3.7-linux-x86_64.egg/pytraj/utils/decorators.py", line 11 in inner
  File "/home/jenkins/jenkins-cpu/workspace/amber-github/pytraj/tests/test_crdframes.py", line 27 in test_crdframes
  File "/opt/conda/lib/python3.7/unittest/case.py", line 628 in run
  File "/opt/conda/lib/python3.7/unittest/case.py", line 676 in __call__
  File "/opt/conda/lib/python3.7/site-packages/_pytest/unittest.py", line 333 in runtest
  File "/opt/conda/lib/python3.7/site-packages/_pytest/runner.py", line 169 in pytest_runtest_call
  File "/opt/conda/lib/python3.7/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/opt/conda/lib/python3.7/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/opt/conda/lib/python3.7/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/opt/conda/lib/python3.7/site-packages/_pytest/runner.py", line 262 in <lambda>
  File "/opt/conda/lib/python3.7/site-packages/_pytest/runner.py", line 341 in from_call
  File "/opt/conda/lib/python3.7/site-packages/_pytest/runner.py", line 262 in call_runtest_hook
  File "/opt/conda/lib/python3.7/site-packages/_pytest/runner.py", line 222 in call_and_report
  File "/opt/conda/lib/python3.7/site-packages/_pytest/runner.py", line 133 in runtestprotocol
  File "/opt/conda/lib/python3.7/site-packages/_pytest/runner.py", line 114 in pytest_runtest_protocol
  File "/opt/conda/lib/python3.7/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/opt/conda/lib/python3.7/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/opt/conda/lib/python3.7/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/opt/conda/lib/python3.7/site-packages/_pytest/main.py", line 348 in pytest_runtestloop
  File "/opt/conda/lib/python3.7/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/opt/conda/lib/python3.7/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/opt/conda/lib/python3.7/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/opt/conda/lib/python3.7/site-packages/_pytest/main.py", line 323 in _main
  File "/opt/conda/lib/python3.7/site-packages/_pytest/main.py", line 269 in wrap_session
  File "/opt/conda/lib/python3.7/site-packages/_pytest/main.py", line 316 in pytest_cmdline_main
  File "/opt/conda/lib/python3.7/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/opt/conda/lib/python3.7/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/opt/conda/lib/python3.7/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/opt/conda/lib/python3.7/site-packages/_pytest/config/__init__.py", line 167 in main
  File "/opt/conda/lib/python3.7/site-packages/_pytest/config/__init__.py", line 189 in console_main
  File "/opt/conda/bin/pytest", line 8 in <module>
Segmentation fault (core dumped)
drroe commented 1 year ago

@hainm so the issue with the randomizeions test (https://github.com/Amber-MD/pytraj/blob/a73c55a306fbf4a06b128a5a640a6929c460e622/tests/test_analysis/test_randomize_ions.py) is related to this code: https://github.com/Amber-MD/pytraj/blob/a73c55a306fbf4a06b128a5a640a6929c460e622/pytraj/trajectory/trajectory_iterator.py#L205-L223

On my local box, I hit the first if isinstance(filename, str) and os.path.exists(filename) case with each iterload call in the test. When Jenkins fails, it's because the second iterload is hitting the second elif isinstance(filename, str) and not os.path.exists(filename) case for some reason. I don't think it's because the pathjoin is failing, because then I think the first iterload would fail too. I can't reproduce it (I'm using python 3.7) no matter what I do. Any ideas? @swails ?

Failed output for reference: https://jenkins.jasonswails.com/job/amber-github/job/pytraj/605/console

drroe commented 1 year ago

@hainm I just noticed that on the Pytraj repo itself, there are a lot of --ignores being passed to pytest which seems to result in the randomize ions test not being run (https://github.com/Amber-MD/pytraj/actions/runs/5446842047/jobs/9908130029#step:10:1). According to the logs it seems to have always been done this way. Why is that?

hainm commented 1 year ago

result in the randomize ions test not being run

Hi Dan, That test has been being run: test_analysis/test_randomize_ions.py::TestRandomizeIons::test_randomize_ions PASSED

According to the logs it seems to have always been done this way. Why is that?

The command was copied from the command running on Jenkin: https://github.com/Amber-MD/pytraj/blob/63d09100066380201ba50aada3da6b72b8e48d24/Jenkinsfile#L33

And the reason pytraj works that way is to make Jenkins happy (e.g: timeout issue with pytraj tests).

Basically it was not easy to debug thing in Jenkins so I tended to skip the affected test. 😅 Let me know if I understand your question correctly.

drroe commented 1 year ago

That test has been being run: test_analysis/test_randomize_ions.py::TestRandomizeIons::test_randomize_ions PASSED

Where are you seeing this? I'm talking about on GitHub Actions, e.g.:

https://github.com/Amber-MD/pytraj/actions/runs/5446842047/jobs/9908130029#step:10:1

I don't see the randomize ions test in any of the output.

drroe commented 1 year ago

@hainm my bad, I needed to use the built-in search, not the browser search. I see it now.

Still no idea why it passes on Actions but fails on Jenkins. Maybe the solution is to skip it on Jenkins? No idea how to really debug it at this point.

drroe commented 1 year ago

I'm going to consider this resolved by https://github.com/Amber-MD/cpptraj/pull/1035. Now that pytraj testing is done via GitHub actions, hopefully the above issues will no longer be a problem.