Closed drew-parsons closed 4 years ago
I can't reproduce the failure in the following environment
pip install duecredit
)I'll check other configurations.
However, can you try updating duecredit as the first obvious difference (even though I doubt that this is the problem, that's one cause to check).
Output
pytest --disable-pytest-warnings --pyargs MDAnalysisTests -k test_data
========================================================= test session starts ==========================================================
platform darwin -- Python 3.8.5, pytest-6.0.2, py-1.9.0, pluggy-0.13.1
rootdir: ~/tmp/mda
plugins: hypothesis-5.28.0
collected 16326 items / 16308 deselected / 2 skipped / 16 selected
analysis/test_data.py . [ 5%]
auxiliary/test_xvg.py .... [ 27%]
coordinates/test_dlpoly.py . [ 33%]
coordinates/test_dms.py . [ 38%]
coordinates/test_gro.py . [ 44%]
coordinates/test_lammps.py .. [ 55%]
coordinates/test_timestep_api.py . [ 61%]
coordinates/test_trj.py . [ 66%]
coordinates/test_trz.py . [ 72%]
utils/test_datafiles.py ... [ 88%]
utils/test_imports.py .. [100%]
====================================== 18 passed, 2 skipped, 16308 deselected, 1 warning in 8.74s ======================================
DueCredit Report:
- Molecular simulation analysis library / MDAnalysis (v 1.0) [1, 2]
- ENCORE Ensemble Comparison / MDAnalysis.analysis.encore (v 1.0) [3]
- Hydrogen bonding autocorrelation time / MDAnalysis.analysis.hbonds.hbond_autocorrel (v 1.0) [4]
- HOLE program, HOLE trajectory analysis with orderparameters / MDAnalysis.analysis.hole (v 1.0) [5, 6, 7]
- Hydrogen bond analysis implementation / MDAnalysis.analysis.hydrogenbonds.hbond_analysis (v 1.0) [8]
- LeafletFinder algorithm / MDAnalysis.analysis.leaflet (v 1.0) [2]
- Path Similarity Analysis algorithm and implementation / MDAnalysis.analysis.psa (v 1.0) [9]
- QCProt implementation / MDAnalysis.lib.qcprot (v 1.0) [10, 11]
- Description of recent updates and new functionality in MDAnalysis since release 0.7.2 (March 2011) up to release 0.16.0 (April 2017). Please cite together with Michaud-Agrawal et al 2011. , Description of MDAnalysis up to release 0.7.2 (March 2011), including the basic philosophy of the library and the LeafletFinder algorithm. Please cite together with Gowers et al 2016. / MDAnalysis/ (v None) [12, 2]
- Scientific tools library / numpy (v 1.14.2) [13]
3 packages cited
7 modules cited
0 functions cited
References
----------
[1] Gowers, R. et al., 2016. MDAnalysis: A Python Package for the Rapid Analysis of Molecular Dynamics Simulations. In Proceedings of the 15th Python in Science Conference. SciPy.
[2] Michaud-Agrawal, N. et al., 2011. MDAnalysis: A toolkit for the analysis of molecular dynamics simulations. Journal of Computational Chemistry, 32(10), pp.2319–2327.
[3] Tiberti, M. et al., 2015. ENCORE: Software for Quantitative Ensemble Comparison B. L. de Groot, ed.. PLOS Computational Biology, 11(10), p.1004415.
[4] Gowers, R.J. & Carbone, P., 2015. A multiscale approach to model hydrogen bonding: The case of polyamide. The Journal of Chemical Physics, 142(22), p.224907.
[5] Smart, O.S., Goodfellow, J.M. & Wallace, B.A., 1993. The pore dimensions of gramicidin A. Biophysical Journal, 65(6), pp.2455–2460.
[6] Smart, O.S. et al., 1996. HOLE: A program for the analysis of the pore dimensions of ion channel structural models. Journal of Molecular Graphics, 14(6), pp.354–360.
[7] Stelzl, L.S. et al., 2014. Flexible Gates Generate Occluded Intermediates in the Transport Cycle of LacY. Journal of Molecular Biology, 426(3), pp.735–751.
[8] Smith, P. et al., 2019. On the interaction of hyaluronic acid with synovial fluid lipid membranes. Physical Chemistry Chemical Physics, 21(19), pp.9845–9857.
[9] Seyler, S.L. et al., 2015. Path Similarity Analysis: A Method for Quantifying Macromolecular Pathways E. Tajkhorshid, ed.. PLOS Computational Biology, 11(10), p.1004568.
[10] Theobald, D.L., 2005. Rapid calculation of RMSDs using a quaternion-based characteristic polynomial. Acta Crystallographica Section A Foundations of Crystallography, 61(4), pp.478–480.
[11] Liu, P., Agrafiotis, D.K. & Theobald, D.L., Fast determination of the optimal rotational matrix for macromolecular superpositions. Journal of Computational Chemistry, 31(7), pp.1561–1563.
[12] Gowers, R.J. et al., 2016. MDAnalysis: A Python package for the rapid analysis of molecular dynamics simulations. In S. Benthall & S. Rostrup, eds. Proceedings of the 15th Python in Science Conference. pp. 102–109.
[13] Van Der Walt, S., Colbert, S.C. & Varoquaux, G., 2011. The NumPy array: a structure for efficient numerical computation. Computing in Science & Engineering, 13(2), pp.22–30.
Running it on the checked-out tests also works:
cd testsuite
rm .duecredit.p
pytest --disable-pytest-warnings -k test_data.py MDAnalysisTests
with output
========================================================= test session starts ==========================================================
platform darwin -- Python 3.8.5, pytest-6.0.2, py-1.9.0, pluggy-0.13.1
rootdir: ~/mdanalysis/testsuite, configfile: setup.cfg
plugins: hypothesis-5.28.0
collected 16390 items / 16388 deselected / 2 skipped
MDAnalysisTests/analysis/test_data.py . [ 50%]
MDAnalysisTests/utils/test_imports.py . [100%]
====================================== 2 passed, 2 skipped, 16388 deselected, 8 warnings in 7.69s ======================================
DueCredit Report:
- Molecular simulation analysis library / MDAnalysis (v 1.0) [1, 2]
- ENCORE Ensemble Comparison / MDAnalysis.analysis.encore (v 1.0) [3]
- Hydrogen bonding autocorrelation time / MDAnalysis.analysis.hbonds.hbond_autocorrel (v 1.0) [4]
- HOLE trajectory analysis with orderparameters, HOLE program / MDAnalysis.analysis.hole (v 1.0) [5, 6, 7]
- Hydrogen bond analysis implementation / MDAnalysis.analysis.hydrogenbonds.hbond_analysis (v 1.0) [8]
- LeafletFinder algorithm / MDAnalysis.analysis.leaflet (v 1.0) [2]
- Path Similarity Analysis algorithm and implementation / MDAnalysis.analysis.psa (v 1.0) [9]
- QCProt implementation / MDAnalysis.lib.qcprot (v 1.0) [10, 11]
- Scientific tools library / numpy (v 1.19.1) [12]
2 packages cited
7 modules cited
0 functions cited
References
----------
[1] Gowers, R. et al., 2016. MDAnalysis: A Python Package for the Rapid Analysis of Molecular Dynamics Simulations. In Proceedings of the 15th Python in Science Conference. SciPy.
[2] Michaud-Agrawal, N. et al., 2011. MDAnalysis: A toolkit for the analysis of molecular dynamics simulations. Journal of Computational Chemistry, 32(10), pp.2319–2327.
[3] Tiberti, M. et al., 2015. ENCORE: Software for Quantitative Ensemble Comparison B. L. de Groot, ed.. PLOS Computational Biology, 11(10), p.1004415.
[4] Gowers, R.J. & Carbone, P., 2015. A multiscale approach to model hydrogen bonding: The case of polyamide. The Journal of Chemical Physics, 142(22), p.224907.
[5] Smart, O.S., Goodfellow, J.M. & Wallace, B.A., 1993. The pore dimensions of gramicidin A. Biophysical Journal, 65(6), pp.2455–2460.
[6] Smart, O.S. et al., 1996. HOLE: A program for the analysis of the pore dimensions of ion channel structural models. Journal of Molecular Graphics, 14(6), pp.354–360.
[7] Stelzl, L.S. et al., 2014. Flexible Gates Generate Occluded Intermediates in the Transport Cycle of LacY. Journal of Molecular Biology, 426(3), pp.735–751.
[8] Smith, P. et al., 2019. On the interaction of hyaluronic acid with synovial fluid lipid membranes. Physical Chemistry Chemical Physics, 21(19), pp.9845–9857.
[9] Seyler, S.L. et al., 2015. Path Similarity Analysis: A Method for Quantifying Macromolecular Pathways E. Tajkhorshid, ed.. PLOS Computational Biology, 11(10), p.1004568.
[10] Theobald, D.L., 2005. Rapid calculation of RMSDs using a quaternion-based characteristic polynomial. Acta Crystallographica Section A Foundations of Crystallography, 61(4), pp.478–480.
[11] Liu, P., Agrafiotis, D.K. & Theobald, D.L., Fast determination of the optimal rotational matrix for macromolecular superpositions. Journal of Computational Chemistry, 31(7), pp.1561–1563.
[12] Van Der Walt, S., Colbert, S.C. & Varoquaux, G., 2011. The NumPy array: a structure for efficient numerical computation. Computing in Science & Engineering, 13(2), pp.22–30.
Are you able to capture the temporary bib files to see what is in the ones that fail?
Does the error persist when you run on another day? The duecredit collector uses the DOI to dynamically resolve the citation. Possibly removing all .duecredit.p
files might also help — find . -name .duecredit.p -print0 | xargs -0 rm
or similar.
Thanks @orbeckst, I'll try some of those variations and see what I can uncover.
The number of citations varies from test to test which is a bit odd. Van Der Walt et all (2011) is always last but between your and my tests it jumps about from [14] to [12] to [13]. Today only 3 citations are failing for me, not 4. Seems today my system is not trying to cite Yang (2008) and the 4th broken entry
Here's a capture of the three "broken" .bibs:
==> tmpc8scs4bk.bib <==
@article{qcprot2,
author = {Pu Liu and Dimitris K. Agrafiotis and Douglas L. Theobald},
title = {Fast determination of the optimal rotational matrix for macromolecular superpositions},
journal = {Journal of Computational Chemistry},
volume = {31},
number = {7},
pages = {1561-1563},
doi = {10.1002/jcc.21439},
}
==> tmpj4pzgf0h.bib <==
@article{Tiberti_2015,
doi = {10.1371/journal.pcbi.1004415},
url = {https://doi.org/10.1371%2Fjournal.pcbi.1004415},
year = 2015,
month = {oct},
publisher = {Public Library of Science ({PLoS})},
volume = {11},
number = {10},
pages = {1004415},
author = {Matteo Tiberti and Elena Papaleo and Tone Bengtsen and Wouter Boomsma and Kresten Lindorff-Larsen},
editor = {Bert L. de Groot},
title = {{ENCORE}: Software for Quantitative Ensemble Comparison},
journal = {{PLOS} Computational Biology}
}
==> tmpy1f_6v02.bib <==
@article{Seyler_2015,
doi = {10.1371/journal.pcbi.1004568},
url = {https://doi.org/10.1371%2Fjournal.pcbi.1004568},
year = 2015,
month = {oct},
publisher = {Public Library of Science ({PLoS})},
volume = {11},
number = {10},
pages = {1004568},
author = {Sean L. Seyler and Avishek Kumar and M. F. Thorpe and Oliver Beckstein},
editor = {Emad Tajkhorshid},
title = {Path Similarity Analysis: A Method for Quantifying Macromolecular Pathways},
journal = {{PLOS} Computational Biology}
}
Those three references are still making a problem with fresh source (i.e. with no .duecredit.p present before tests).
Today's citation list (still with duecredit 0.7.0) is
[1] Gowers, R. et al., 2016. MDAnalysis: A Python Package for the Rapid Analysis of Molecular Dynamics Simulations. In Proceedings of the 15th Python in Science Conference. SciPy.
[2] Michaud-Agrawal, N. et al., 2011. MDAnalysis: A toolkit for the analysis of molecular dynamics simulations. Journal of Computational Chemistry, 32(10), pp.2319–2327.
[3] ERRORED: not enough values to unpack (expected 2, got 1)
[4] Gowers, R.J. & Carbone, P., 2015. A multiscale approach to model hydrogen bonding: The case of polyamide. The Journal of Chemical Physics, 142(22), p.224907.
[5] Smart, O.S., Goodfellow, J.M. & Wallace, B.A., 1993. The pore dimensions of gramicidin A. Biophysical Journal, 65(6), pp.2455–2460.
[6] Smart, O.S. et al., 1996. HOLE: A program for the analysis of the pore dimensions of ion channel structural models. Journal of Molecular Graphics, 14(6), pp.354–360.
[7] Stelzl, L.S. et al., 2014. Flexible Gates Generate Occluded Intermediates in the Transport Cycle of LacY. Journal of Molecular Biology, 426(3), pp.735–751.
[8] Smith, P. et al., 2019. On the interaction of hyaluronic acid with synovial fluid lipid membranes. Physical Chemistry Chemical Physics, 21(19), pp.9845–9857.
[9] ERRORED: not enough values to unpack (expected 2, got 1)
[10] Theobald, D.L., 2005. Rapid calculation of RMSDs using a quaternion-based characteristic polynomial. Acta Crystallographica Section A Foundations of Crystallography, 61(4), pp.478–480.
[11] ERRORED: not enough values to unpack (expected 2, got 1)
[12] Van Der Walt, S., Colbert, S.C. & Varoquaux, G., 2011. The NumPy array: a structure for efficient numerical computation. Computing in Science & Engineering, 13(2), pp.22–30.
Sorry for troubles with duecredit. value.replace(' ', '').split('--')
issue I believe is due to citeproc, I believe it was fixed since awhile. citeproc in debian is abit outdated, I am looking at updating it now to 0.5.1 now that rcn2rng is there as well. Meanwhile if you could test that upgrade of duecredit + citeproc to the latests in pypi would help -- that would be great and appreciated.
Citations are now clean with duecredit 0.8.0 (aided and abetted by citeproc 0.5.1). Thanks @orbeckst, thanks @yarikoptic
I'll close this issue then.
p.s. pytest-3 -k test_data.py
is now running citations cleanly, but a full run with pytest
(via Debian's autopkgtest -B -- null
) gives one more glitch:
...
[9] Yang, L. et al., 2008. Close Correspondence between the Motions from Principal Component Analysis of Multiple HIV-1 Protease Structures and Elastic Network Modes. Structure, 16(2), pp.321–330.
[10] Seyler, S.L. et al., 2015. Path Similarity Analysis: A Method for Quantifying Macromolecular Pathways E. Tajkhorshid, ed.. PLOS Computational Biology, 11(10), p.1004568.
[11] ERRORED: generator raised StopIteration
[12] Theobald, D.L., 2005. Rapid calculation of RMSDs using a quaternion-based characteristic polynomial. Acta Crystallographica Section A Foundations of Crystallography, 61(4), pp.478–480.
[13] Liu, P., Agrafiotis, D.K. & Theobald, D.L., Fast determination of the optimal rotational matrix for macromolecular superpositions. Journal of Computational Chemistry, 31(7), pp.1561–1563.
[14] Van Der Walt, S., Colbert, S.C. & Varoquaux, G., 2011. The NumPy array: a structure for efficient numerical computation. Computing in Science & Engineering, 13(2), pp.22–30.
Exception ignored in: <function ReaderBase.__del__ at 0x7f80484ee550>
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/MDAnalysis/coordinates/base.py", line 2152, in __del__
self.close()
File "/usr/lib/python3/dist-packages/MDAnalysis/coordinates/PDB.py", line 462, in close
self._pdbfile.close()
File "/usr/lib/python3/dist-packages/MDAnalysis/lib/util.py", line 739, in close
self.flush()
File "/usr/lib/python3/dist-packages/MDAnalysis/lib/util.py", line 845, in flush
return self.stream.flush()
ValueError: I/O operation on closed file.
The short run on a single test only generates 11 citations, not the 14 cited here. Must be some other test bringing the algorithm cited in this broken [11]. Possibly not worth reopening this bug for it right now, but something to keep an eye on.
I'll file a bug once the mdanalysis package is accepted into the Debian archives, if it's still happening then.
It's possible that the test does not collect all citations. E.g., the ENCORE citations are only collected when MDAnalysis.analysis.encore is imported (IIRC).
The ValueError: I/O operation on closed file.
is harmless (but admittedly not pretty).
Expected behavior
MDAnalysis has been configured so that when pytest is run, it runs duecredit after completion. The references reported by duecredit should be cited cleanly without triggering a reporting error.
Actual behavior
When pytest is launched, all tests successfully run (or skip/xfail as needed), no fails. After completing tests, MDAnalysis launches a duecredit report. This fails, apparently due to misconfigured reference entries.
The pytest run itself returns error code 0, so the problem is not registering as an "error" as such, as far as pytest goes. But the report of
ERRORED: not enough values to unpack
is disconcerting and the Logging error does go to stderr so could be interpreted by some systems as a test fail.Any variation of pytest (including
pytest --collect-only
) triggers the duecredit report with the same broken citations. Test output on test_data.py only (selected since it has only 1 test) is:It might be easier to see which citations are causing the problem by hiding stderr:
Code to reproduce the behavior
(on Debian, that's
pytest-3 -k test_data.py
)or any other variation of
pytest
.Current version of MDAnalysis
python -V
)? Python 3.8.6rc1