MDAnalysis / mdanalysis

MDAnalysis is a Python library to analyze molecular dynamics simulations.
https://mdanalysis.org
Other
1.32k stars 652 forks source link

duecredit references broken giving citation report error #2954

Closed drew-parsons closed 4 years ago

drew-parsons commented 4 years ago

Expected behavior

MDAnalysis has been configured so that when pytest is run, it runs duecredit after completion. The references reported by duecredit should be cited cleanly without triggering a reporting error.

Actual behavior

When pytest is launched, all tests successfully run (or skip/xfail as needed), no fails. After completing tests, MDAnalysis launches a duecredit report. This fails, apparently due to misconfigured reference entries.

The pytest run itself returns error code 0, so the problem is not registering as an "error" as such, as far as pytest goes. But the report of ERRORED: not enough values to unpack is disconcerting and the Logging error does go to stderr so could be interpreted by some systems as a test fail.

Any variation of pytest (including pytest --collect-only) triggers the duecredit report with the same broken citations. Test output on test_data.py only (selected since it has only 1 test) is:

$ pytest-3  -k test_data.py
=============================================================================================== test session starts ===============================================================================================
platform linux -- Python 3.8.6rc1, pytest-4.6.11, py-1.9.0, pluggy-0.13.0
rootdir: /home/drew/projects/debichem/build/mdanalysis
plugins: forked-1.3.0, xdist-1.32.0, doctestplus-0.7.0, astropy-header-0.1.2, asyncio-0.14.0, hypothesis-5.32.1, filter-subpackage-0.1.1, cov-2.8.1, remotedata-0.3.2, mpi-0+unknown, arraydiff-0.3, openfiles-0.5.0
collected 16238 items / 16236 deselected / 3 skipped                                                                                                                                                              

testsuite/MDAnalysisTests/analysis/test_data.py .                                                                                                                                                           [ 50%]
testsuite/MDAnalysisTests/utils/test_imports.py .                                                                                                                                                           [100%]

================================================================================================ warnings summary =================================================================================================
/usr/lib/python3/dist-packages/duecredit/io.py:18
  /usr/lib/python3/dist-packages/duecredit/io.py:18: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working
    from collections import defaultdict, Iterator

/usr/lib/python3/dist-packages/MDAnalysis/analysis/hbonds/hbond_analysis.py:342
  /usr/lib/python3/dist-packages/MDAnalysis/analysis/hbonds/hbond_analysis.py:342: DeprecationWarning: This module is deprecated as of MDAnalysis version 1.0.It will be removed in MDAnalysis version 2.0Please use MDAnalysis.analysis.hydrogenbonds.hbond_analysis instead.
    warnings.warn(

-- Docs: https://docs.pytest.org/en/latest/warnings.html
======================================================================== 2 passed, 3 skipped, 16236 deselected, 2 warnings in 4.67 seconds ========================================================================

DueCredit Report:
- Molecular simulation analysis library / MDAnalysis (v 1.0) [1, 2]
  - ENCORE Ensemble Comparison / MDAnalysis.analysis.encore (v 1.0) [3]
  - Hydrogen bonding autocorrelation time / MDAnalysis.analysis.hbonds.hbond_autocorrel (v 1.0) [4]
  - HOLE trajectory analysis with orderparameters, HOLE program / MDAnalysis.analysis.hole (v 1.0) [5, 6, 7]
  - Hydrogen bond analysis implementation / MDAnalysis.analysis.hydrogenbonds.hbond_analysis (v 1.0) [8]
  - LeafletFinder algorithm / MDAnalysis.analysis.leaflet (v 1.0) [2]
  - Cumulative overlap / MDAnalysis.analysis.pca (v 1.0) [9]
  - Path Similarity Analysis algorithm and implementation / MDAnalysis.analysis.psa (v 1.0) [10]
  - MMTF Reader / MDAnalysis.coordinates.MMTF (v 1.0) [11]
  - QCProt implementation / MDAnalysis.lib.qcprot (v 1.0) [12, 13]
  - MMTF Parser / MDAnalysis.topology.MMTFParser (v 1.0) [11]
- Scientific tools library / numpy (v 1.19.2) [14]

2 packages cited
10 modules cited
0 functions cited

References
----------

[1] Gowers, R. et al., 2016. MDAnalysis: A Python Package for the Rapid Analysis of Molecular Dynamics Simulations. In Proceedings of the 15th Python in Science Conference.  SciPy.
[2] Michaud-Agrawal, N. et al., 2011. MDAnalysis: A toolkit for the analysis of molecular dynamics simulations. Journal of Computational Chemistry, 32(10), pp.2319–2327.
[3] --- Logging error ---
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/duecredit/io.py", line 306, in format_bibtex
    bib_source = cpBibTeX(fname)
  File "/usr/lib/python3/dist-packages/citeproc/source/bibtex/bibtex.py", line 71, in __init__
    self.add(self.create_reference(key, entry))
  File "/usr/lib/python3/dist-packages/citeproc/source/bibtex/bibtex.py", line 202, in create_reference
    csl_fields = self._bibtex_to_csl(bibtex_entry)
  File "/usr/lib/python3/dist-packages/citeproc/source/bibtex/bibtex.py", line 91, in _bibtex_to_csl
    first, last = value.replace(' ', '').split('--')
ValueError: not enough values to unpack (expected 2, got 1)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.8/logging/__init__.py", line 1084, in emit
    stream.write(msg + self.terminator)
  File "/usr/lib/python3/dist-packages/_pytest/capture.py", line 441, in write
    self.buffer.write(obj)
ValueError: I/O operation on closed file
Call stack:
  File "/usr/lib/python3/dist-packages/duecredit/utils.py", line 195, in wrapped_func
    return f(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/duecredit/dueswitch.py", line 82, in dump
    due_summary.dump()
  File "/usr/lib/python3/dist-packages/duecredit/collector.py", line 470, in dump
    output.dump()
  File "/usr/lib/python3/dist-packages/duecredit/io.py", line 222, in dump
    self.fd.write(get_text_rendering(cit,
  File "/usr/lib/python3/dist-packages/duecredit/io.py", line 235, in get_text_rendering
    return get_text_rendering(bibtex_citation)
  File "/usr/lib/python3/dist-packages/duecredit/io.py", line 237, in get_text_rendering
    return format_bibtex(entry, style=style)
  File "/usr/lib/python3/dist-packages/duecredit/io.py", line 313, in format_bibtex
    lgr.error("Failed to process BibTeX file %s: %s" % (fname, e))
Message: 'Failed to process BibTeX file /tmp/tmpxh7unwwz.bib: not enough values to unpack (expected 2, got 1)'
Arguments: ()
ERRORED: not enough values to unpack (expected 2, got 1)
[4] Gowers, R.J. & Carbone, P., 2015. A multiscale approach to model hydrogen bonding: The case of polyamide. The Journal of Chemical Physics, 142(22), p.224907.
[5] Smart, O.S., Goodfellow, J.M. & Wallace, B.A., 1993. The pore dimensions of gramicidin A. Biophysical Journal, 65(6), pp.2455–2460.
[6] Smart, O.S. et al., 1996. HOLE: A program for the analysis of the pore dimensions of ion channel structural models. Journal of Molecular Graphics, 14(6), pp.354–360.
[7] Stelzl, L.S. et al., 2014. Flexible Gates Generate Occluded Intermediates in the Transport Cycle of LacY. Journal of Molecular Biology, 426(3), pp.735–751.
[8] Smith, P. et al., 2019. On the interaction of hyaluronic acid with synovial fluid lipid membranes. Physical Chemistry Chemical Physics, 21(19), pp.9845–9857.
[9] Yang, L. et al., 2008. Close Correspondence between the Motions from Principal Component Analysis of Multiple HIV-1 Protease Structures and Elastic Network Modes. Structure, 16(2), pp.321–330.
[10] --- Logging error ---
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/duecredit/io.py", line 306, in format_bibtex
    bib_source = cpBibTeX(fname)
  File "/usr/lib/python3/dist-packages/citeproc/source/bibtex/bibtex.py", line 71, in __init__
    self.add(self.create_reference(key, entry))
  File "/usr/lib/python3/dist-packages/citeproc/source/bibtex/bibtex.py", line 202, in create_reference
    csl_fields = self._bibtex_to_csl(bibtex_entry)
  File "/usr/lib/python3/dist-packages/citeproc/source/bibtex/bibtex.py", line 91, in _bibtex_to_csl
    first, last = value.replace(' ', '').split('--')
ValueError: not enough values to unpack (expected 2, got 1)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.8/logging/__init__.py", line 1084, in emit
    stream.write(msg + self.terminator)
  File "/usr/lib/python3/dist-packages/_pytest/capture.py", line 441, in write
    self.buffer.write(obj)
ValueError: I/O operation on closed file
Call stack:
  File "/usr/lib/python3/dist-packages/duecredit/utils.py", line 195, in wrapped_func
    return f(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/duecredit/dueswitch.py", line 82, in dump
    due_summary.dump()
  File "/usr/lib/python3/dist-packages/duecredit/collector.py", line 470, in dump
    output.dump()
  File "/usr/lib/python3/dist-packages/duecredit/io.py", line 222, in dump
    self.fd.write(get_text_rendering(cit,
  File "/usr/lib/python3/dist-packages/duecredit/io.py", line 235, in get_text_rendering
    return get_text_rendering(bibtex_citation)
  File "/usr/lib/python3/dist-packages/duecredit/io.py", line 237, in get_text_rendering
    return format_bibtex(entry, style=style)
  File "/usr/lib/python3/dist-packages/duecredit/io.py", line 313, in format_bibtex
    lgr.error("Failed to process BibTeX file %s: %s" % (fname, e))
Message: 'Failed to process BibTeX file /tmp/tmpui17n1le.bib: not enough values to unpack (expected 2, got 1)'
Arguments: ()
ERRORED: not enough values to unpack (expected 2, got 1)
[11] --- Logging error ---
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/duecredit/io.py", line 306, in format_bibtex
    bib_source = cpBibTeX(fname)
  File "/usr/lib/python3/dist-packages/citeproc/source/bibtex/bibtex.py", line 71, in __init__
    self.add(self.create_reference(key, entry))
  File "/usr/lib/python3/dist-packages/citeproc/source/bibtex/bibtex.py", line 202, in create_reference
    csl_fields = self._bibtex_to_csl(bibtex_entry)
  File "/usr/lib/python3/dist-packages/citeproc/source/bibtex/bibtex.py", line 91, in _bibtex_to_csl
    first, last = value.replace(' ', '').split('--')
ValueError: not enough values to unpack (expected 2, got 1)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.8/logging/__init__.py", line 1084, in emit
    stream.write(msg + self.terminator)
  File "/usr/lib/python3/dist-packages/_pytest/capture.py", line 441, in write
    self.buffer.write(obj)
ValueError: I/O operation on closed file
Call stack:
  File "/usr/lib/python3/dist-packages/duecredit/utils.py", line 195, in wrapped_func
    return f(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/duecredit/dueswitch.py", line 82, in dump
    due_summary.dump()
  File "/usr/lib/python3/dist-packages/duecredit/collector.py", line 470, in dump
    output.dump()
  File "/usr/lib/python3/dist-packages/duecredit/io.py", line 222, in dump
    self.fd.write(get_text_rendering(cit,
  File "/usr/lib/python3/dist-packages/duecredit/io.py", line 235, in get_text_rendering
    return get_text_rendering(bibtex_citation)
  File "/usr/lib/python3/dist-packages/duecredit/io.py", line 237, in get_text_rendering
    return format_bibtex(entry, style=style)
  File "/usr/lib/python3/dist-packages/duecredit/io.py", line 313, in format_bibtex
    lgr.error("Failed to process BibTeX file %s: %s" % (fname, e))
Message: 'Failed to process BibTeX file /tmp/tmp68nrvtq1.bib: not enough values to unpack (expected 2, got 1)'
Arguments: ()
ERRORED: not enough values to unpack (expected 2, got 1)
[12] Theobald, D.L., 2005. Rapid calculation of RMSDs using a quaternion-based characteristic polynomial. Acta Crystallographica Section A Foundations of Crystallography, 61(4), pp.478–480.
[13] --- Logging error ---
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/duecredit/io.py", line 306, in format_bibtex
    bib_source = cpBibTeX(fname)
  File "/usr/lib/python3/dist-packages/citeproc/source/bibtex/bibtex.py", line 71, in __init__
    self.add(self.create_reference(key, entry))
  File "/usr/lib/python3/dist-packages/citeproc/source/bibtex/bibtex.py", line 202, in create_reference
    csl_fields = self._bibtex_to_csl(bibtex_entry)
  File "/usr/lib/python3/dist-packages/citeproc/source/bibtex/bibtex.py", line 91, in _bibtex_to_csl
    first, last = value.replace(' ', '').split('--')
ValueError: not enough values to unpack (expected 2, got 1)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.8/logging/__init__.py", line 1084, in emit
    stream.write(msg + self.terminator)
  File "/usr/lib/python3/dist-packages/_pytest/capture.py", line 441, in write
    self.buffer.write(obj)
ValueError: I/O operation on closed file
Call stack:
  File "/usr/lib/python3/dist-packages/duecredit/utils.py", line 195, in wrapped_func
    return f(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/duecredit/dueswitch.py", line 82, in dump
    due_summary.dump()
  File "/usr/lib/python3/dist-packages/duecredit/collector.py", line 470, in dump
    output.dump()
  File "/usr/lib/python3/dist-packages/duecredit/io.py", line 222, in dump
    self.fd.write(get_text_rendering(cit,
  File "/usr/lib/python3/dist-packages/duecredit/io.py", line 237, in get_text_rendering
    return format_bibtex(entry, style=style)
  File "/usr/lib/python3/dist-packages/duecredit/io.py", line 313, in format_bibtex
    lgr.error("Failed to process BibTeX file %s: %s" % (fname, e))
Message: 'Failed to process BibTeX file /tmp/tmpkgkmb976.bib: not enough values to unpack (expected 2, got 1)'
Arguments: ()
ERRORED: not enough values to unpack (expected 2, got 1)
[14] Van Der Walt, S., Colbert, S.C. & Varoquaux, G., 2011. The NumPy array: a structure for efficient numerical computation. Computing in Science & Engineering, 13(2), pp.22–30.

It might be easier to see which citations are causing the problem by hiding stderr:

$ pytest-3  -k test_data.py 2>/dev/null
=============================================================================================== test session starts ===============================================================================================
platform linux -- Python 3.8.6rc1, pytest-4.6.11, py-1.9.0, pluggy-0.13.0
rootdir: /home/drew/projects/debichem/build/mdanalysis
plugins: forked-1.3.0, xdist-1.32.0, doctestplus-0.7.0, astropy-header-0.1.2, asyncio-0.14.0, hypothesis-5.32.1, filter-subpackage-0.1.1, cov-2.8.1, remotedata-0.3.2, mpi-0+unknown, arraydiff-0.3, openfiles-0.5.0
collected 16238 items / 16236 deselected / 3 skipped                                                                                                                                                              

testsuite/MDAnalysisTests/analysis/test_data.py .                                                                                                                                                           [ 50%]
testsuite/MDAnalysisTests/utils/test_imports.py .                                                                                                                                                           [100%]

================================================================================================ warnings summary =================================================================================================
/usr/lib/python3/dist-packages/duecredit/io.py:18
  /usr/lib/python3/dist-packages/duecredit/io.py:18: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working
    from collections import defaultdict, Iterator

/usr/lib/python3/dist-packages/MDAnalysis/analysis/hbonds/hbond_analysis.py:342
  /usr/lib/python3/dist-packages/MDAnalysis/analysis/hbonds/hbond_analysis.py:342: DeprecationWarning: This module is deprecated as of MDAnalysis version 1.0.It will be removed in MDAnalysis version 2.0Please use MDAnalysis.analysis.hydrogenbonds.hbond_analysis instead.
    warnings.warn(

-- Docs: https://docs.pytest.org/en/latest/warnings.html
======================================================================== 2 passed, 3 skipped, 16236 deselected, 2 warnings in 4.79 seconds ========================================================================

DueCredit Report:
- Molecular simulation analysis library / MDAnalysis (v 1.0) [1, 2]
  - ENCORE Ensemble Comparison / MDAnalysis.analysis.encore (v 1.0) [3]
  - Hydrogen bonding autocorrelation time / MDAnalysis.analysis.hbonds.hbond_autocorrel (v 1.0) [4]
  - HOLE program, HOLE trajectory analysis with orderparameters / MDAnalysis.analysis.hole (v 1.0) [5, 6, 7]
  - Hydrogen bond analysis implementation / MDAnalysis.analysis.hydrogenbonds.hbond_analysis (v 1.0) [8]
  - LeafletFinder algorithm / MDAnalysis.analysis.leaflet (v 1.0) [2]
  - Cumulative overlap / MDAnalysis.analysis.pca (v 1.0) [9]
  - Path Similarity Analysis algorithm and implementation / MDAnalysis.analysis.psa (v 1.0) [10]
  - MMTF Reader / MDAnalysis.coordinates.MMTF (v 1.0) [11]
  - QCProt implementation / MDAnalysis.lib.qcprot (v 1.0) [12, 13]
  - MMTF Parser / MDAnalysis.topology.MMTFParser (v 1.0) [11]
- Scientific tools library / numpy (v 1.19.2) [14]

2 packages cited
10 modules cited
0 functions cited

References
----------

[1] Gowers, R. et al., 2016. MDAnalysis: A Python Package for the Rapid Analysis of Molecular Dynamics Simulations. In Proceedings of the 15th Python in Science Conference.  SciPy.
[2] Michaud-Agrawal, N. et al., 2011. MDAnalysis: A toolkit for the analysis of molecular dynamics simulations. Journal of Computational Chemistry, 32(10), pp.2319–2327.
[3] ERRORED: not enough values to unpack (expected 2, got 1)
[4] Gowers, R.J. & Carbone, P., 2015. A multiscale approach to model hydrogen bonding: The case of polyamide. The Journal of Chemical Physics, 142(22), p.224907.
[5] Smart, O.S., Goodfellow, J.M. & Wallace, B.A., 1993. The pore dimensions of gramicidin A. Biophysical Journal, 65(6), pp.2455–2460.
[6] Smart, O.S. et al., 1996. HOLE: A program for the analysis of the pore dimensions of ion channel structural models. Journal of Molecular Graphics, 14(6), pp.354–360.
[7] Stelzl, L.S. et al., 2014. Flexible Gates Generate Occluded Intermediates in the Transport Cycle of LacY. Journal of Molecular Biology, 426(3), pp.735–751.
[8] Smith, P. et al., 2019. On the interaction of hyaluronic acid with synovial fluid lipid membranes. Physical Chemistry Chemical Physics, 21(19), pp.9845–9857.
[9] Yang, L. et al., 2008. Close Correspondence between the Motions from Principal Component Analysis of Multiple HIV-1 Protease Structures and Elastic Network Modes. Structure, 16(2), pp.321–330.
[10] ERRORED: not enough values to unpack (expected 2, got 1)
[11] ERRORED: not enough values to unpack (expected 2, got 1)
[12] Theobald, D.L., 2005. Rapid calculation of RMSDs using a quaternion-based characteristic polynomial. Acta Crystallographica Section A Foundations of Crystallography, 61(4), pp.478–480.
[13] ERRORED: not enough values to unpack (expected 2, got 1)
[14] Van Der Walt, S., Colbert, S.C. & Varoquaux, G., 2011. The NumPy array: a structure for efficient numerical computation. Computing in Science & Engineering, 13(2), pp.22–30.

Code to reproduce the behavior

pytest -k test_data.py

(on Debian, that's pytest-3 -k test_data.py)

or any other variation of pytest.

Current version of MDAnalysis

orbeckst commented 4 years ago

I can't reproduce the failure in the following environment

I'll check other configurations.

However, can you try updating duecredit as the first obvious difference (even though I doubt that this is the problem, that's one cause to check).

Output

 pytest --disable-pytest-warnings --pyargs MDAnalysisTests -k test_data
========================================================= test session starts ==========================================================
platform darwin -- Python 3.8.5, pytest-6.0.2, py-1.9.0, pluggy-0.13.1
rootdir: ~/tmp/mda
plugins: hypothesis-5.28.0
collected 16326 items / 16308 deselected / 2 skipped / 16 selected

analysis/test_data.py .                                                                                                          [  5%]
auxiliary/test_xvg.py ....                                                                                                       [ 27%]
coordinates/test_dlpoly.py .                                                                                                     [ 33%]
coordinates/test_dms.py .                                                                                                        [ 38%]
coordinates/test_gro.py .                                                                                                        [ 44%]
coordinates/test_lammps.py ..                                                                                                    [ 55%]
coordinates/test_timestep_api.py .                                                                                               [ 61%]
coordinates/test_trj.py .                                                                                                        [ 66%]
coordinates/test_trz.py .                                                                                                        [ 72%]
utils/test_datafiles.py ...                                                                                                      [ 88%]
utils/test_imports.py ..                                                                                                         [100%]

====================================== 18 passed, 2 skipped, 16308 deselected, 1 warning in 8.74s ======================================

DueCredit Report:
- Molecular simulation analysis library / MDAnalysis (v 1.0) [1, 2]
  - ENCORE Ensemble Comparison / MDAnalysis.analysis.encore (v 1.0) [3]
  - Hydrogen bonding autocorrelation time / MDAnalysis.analysis.hbonds.hbond_autocorrel (v 1.0) [4]
  - HOLE program, HOLE trajectory analysis with orderparameters / MDAnalysis.analysis.hole (v 1.0) [5, 6, 7]
  - Hydrogen bond analysis implementation / MDAnalysis.analysis.hydrogenbonds.hbond_analysis (v 1.0) [8]
  - LeafletFinder algorithm / MDAnalysis.analysis.leaflet (v 1.0) [2]
  - Path Similarity Analysis algorithm and implementation / MDAnalysis.analysis.psa (v 1.0) [9]
  - QCProt implementation / MDAnalysis.lib.qcprot (v 1.0) [10, 11]
- Description of recent updates and new functionality in MDAnalysis since release 0.7.2 (March 2011) up to release 0.16.0 (April 2017). Please cite together with Michaud-Agrawal et al 2011. , Description of MDAnalysis up to release 0.7.2 (March 2011), including the basic philosophy of the library and the LeafletFinder algorithm. Please cite together with Gowers et al 2016.  / MDAnalysis/ (v None) [12, 2]
- Scientific tools library / numpy (v 1.14.2) [13]

3 packages cited
7 modules cited
0 functions cited

References
----------

[1] Gowers, R. et al., 2016. MDAnalysis: A Python Package for the Rapid Analysis of Molecular Dynamics Simulations. In Proceedings of the 15th Python in Science Conference.  SciPy.
[2] Michaud-Agrawal, N. et al., 2011. MDAnalysis: A toolkit for the analysis of molecular dynamics simulations. Journal of Computational Chemistry, 32(10), pp.2319–2327.
[3] Tiberti, M. et al., 2015. ENCORE: Software for Quantitative Ensemble Comparison B. L. de Groot, ed.. PLOS Computational Biology, 11(10), p.1004415.
[4] Gowers, R.J. & Carbone, P., 2015. A multiscale approach to model hydrogen bonding: The case of polyamide. The Journal of Chemical Physics, 142(22), p.224907.
[5] Smart, O.S., Goodfellow, J.M. & Wallace, B.A., 1993. The pore dimensions of gramicidin A. Biophysical Journal, 65(6), pp.2455–2460.
[6] Smart, O.S. et al., 1996. HOLE: A program for the analysis of the pore dimensions of ion channel structural models. Journal of Molecular Graphics, 14(6), pp.354–360.
[7] Stelzl, L.S. et al., 2014. Flexible Gates Generate Occluded Intermediates in the Transport Cycle of LacY. Journal of Molecular Biology, 426(3), pp.735–751.
[8] Smith, P. et al., 2019. On the interaction of hyaluronic acid with synovial fluid lipid membranes. Physical Chemistry Chemical Physics, 21(19), pp.9845–9857.
[9] Seyler, S.L. et al., 2015. Path Similarity Analysis: A Method for Quantifying Macromolecular Pathways E. Tajkhorshid, ed.. PLOS Computational Biology, 11(10), p.1004568.
[10] Theobald, D.L., 2005. Rapid calculation of RMSDs using a quaternion-based characteristic polynomial. Acta Crystallographica Section A Foundations of Crystallography, 61(4), pp.478–480.
[11] Liu, P., Agrafiotis, D.K. & Theobald, D.L., Fast determination of the optimal rotational matrix for macromolecular superpositions. Journal of Computational Chemistry, 31(7), pp.1561–1563.
[12] Gowers, R.J. et al., 2016. MDAnalysis: A Python package for the rapid analysis of molecular dynamics simulations. In S. Benthall & S. Rostrup, eds. Proceedings of the 15th Python in Science Conference. pp. 102–109.
[13] Van Der Walt, S., Colbert, S.C. & Varoquaux, G., 2011. The NumPy array: a structure for efficient numerical computation. Computing in Science & Engineering, 13(2), pp.22–30.
orbeckst commented 4 years ago

Running it on the checked-out tests also works:

cd testsuite
rm .duecredit.p
pytest --disable-pytest-warnings  -k test_data.py MDAnalysisTests

with output

========================================================= test session starts ==========================================================
platform darwin -- Python 3.8.5, pytest-6.0.2, py-1.9.0, pluggy-0.13.1
rootdir: ~/mdanalysis/testsuite, configfile: setup.cfg
plugins: hypothesis-5.28.0
collected 16390 items / 16388 deselected / 2 skipped

MDAnalysisTests/analysis/test_data.py .                                                                                          [ 50%]
MDAnalysisTests/utils/test_imports.py .                                                                                          [100%]

====================================== 2 passed, 2 skipped, 16388 deselected, 8 warnings in 7.69s ======================================

DueCredit Report:
- Molecular simulation analysis library / MDAnalysis (v 1.0) [1, 2]
  - ENCORE Ensemble Comparison / MDAnalysis.analysis.encore (v 1.0) [3]
  - Hydrogen bonding autocorrelation time / MDAnalysis.analysis.hbonds.hbond_autocorrel (v 1.0) [4]
  - HOLE trajectory analysis with orderparameters, HOLE program / MDAnalysis.analysis.hole (v 1.0) [5, 6, 7]
  - Hydrogen bond analysis implementation / MDAnalysis.analysis.hydrogenbonds.hbond_analysis (v 1.0) [8]
  - LeafletFinder algorithm / MDAnalysis.analysis.leaflet (v 1.0) [2]
  - Path Similarity Analysis algorithm and implementation / MDAnalysis.analysis.psa (v 1.0) [9]
  - QCProt implementation / MDAnalysis.lib.qcprot (v 1.0) [10, 11]
- Scientific tools library / numpy (v 1.19.1) [12]

2 packages cited
7 modules cited
0 functions cited

References
----------

[1] Gowers, R. et al., 2016. MDAnalysis: A Python Package for the Rapid Analysis of Molecular Dynamics Simulations. In Proceedings of the 15th Python in Science Conference.  SciPy.
[2] Michaud-Agrawal, N. et al., 2011. MDAnalysis: A toolkit for the analysis of molecular dynamics simulations. Journal of Computational Chemistry, 32(10), pp.2319–2327.
[3] Tiberti, M. et al., 2015. ENCORE: Software for Quantitative Ensemble Comparison B. L. de Groot, ed.. PLOS Computational Biology, 11(10), p.1004415.
[4] Gowers, R.J. & Carbone, P., 2015. A multiscale approach to model hydrogen bonding: The case of polyamide. The Journal of Chemical Physics, 142(22), p.224907.
[5] Smart, O.S., Goodfellow, J.M. & Wallace, B.A., 1993. The pore dimensions of gramicidin A. Biophysical Journal, 65(6), pp.2455–2460.
[6] Smart, O.S. et al., 1996. HOLE: A program for the analysis of the pore dimensions of ion channel structural models. Journal of Molecular Graphics, 14(6), pp.354–360.
[7] Stelzl, L.S. et al., 2014. Flexible Gates Generate Occluded Intermediates in the Transport Cycle of LacY. Journal of Molecular Biology, 426(3), pp.735–751.
[8] Smith, P. et al., 2019. On the interaction of hyaluronic acid with synovial fluid lipid membranes. Physical Chemistry Chemical Physics, 21(19), pp.9845–9857.
[9] Seyler, S.L. et al., 2015. Path Similarity Analysis: A Method for Quantifying Macromolecular Pathways E. Tajkhorshid, ed.. PLOS Computational Biology, 11(10), p.1004568.
[10] Theobald, D.L., 2005. Rapid calculation of RMSDs using a quaternion-based characteristic polynomial. Acta Crystallographica Section A Foundations of Crystallography, 61(4), pp.478–480.
[11] Liu, P., Agrafiotis, D.K. & Theobald, D.L., Fast determination of the optimal rotational matrix for macromolecular superpositions. Journal of Computational Chemistry, 31(7), pp.1561–1563.
[12] Van Der Walt, S., Colbert, S.C. & Varoquaux, G., 2011. The NumPy array: a structure for efficient numerical computation. Computing in Science & Engineering, 13(2), pp.22–30.
orbeckst commented 4 years ago

Are you able to capture the temporary bib files to see what is in the ones that fail?

Does the error persist when you run on another day? The duecredit collector uses the DOI to dynamically resolve the citation. Possibly removing all .duecredit.p files might also help — find . -name .duecredit.p -print0 | xargs -0 rm or similar.

drew-parsons commented 4 years ago

Thanks @orbeckst, I'll try some of those variations and see what I can uncover.

drew-parsons commented 4 years ago

The number of citations varies from test to test which is a bit odd. Van Der Walt et all (2011) is always last but between your and my tests it jumps about from [14] to [12] to [13]. Today only 3 citations are failing for me, not 4. Seems today my system is not trying to cite Yang (2008) and the 4th broken entry

Here's a capture of the three "broken" .bibs:


==> tmpc8scs4bk.bib <==
@article{qcprot2,
author = {Pu Liu and Dimitris K. Agrafiotis and Douglas L. Theobald},
title = {Fast determination of the optimal rotational matrix for macromolecular superpositions},
journal = {Journal of Computational Chemistry},
volume = {31},
number = {7},
pages = {1561-1563},
doi = {10.1002/jcc.21439},
}
==> tmpj4pzgf0h.bib <==
@article{Tiberti_2015,
    doi = {10.1371/journal.pcbi.1004415},
    url = {https://doi.org/10.1371%2Fjournal.pcbi.1004415},
    year = 2015,
    month = {oct},
    publisher = {Public Library of Science ({PLoS})},
    volume = {11},
    number = {10},
    pages = {1004415},
    author = {Matteo Tiberti and Elena Papaleo and Tone Bengtsen and Wouter Boomsma and Kresten Lindorff-Larsen},
    editor = {Bert L. de Groot},
    title = {{ENCORE}: Software for Quantitative Ensemble Comparison},
    journal = {{PLOS} Computational Biology}
}
==> tmpy1f_6v02.bib <==
@article{Seyler_2015,
    doi = {10.1371/journal.pcbi.1004568},
    url = {https://doi.org/10.1371%2Fjournal.pcbi.1004568},
    year = 2015,
    month = {oct},
    publisher = {Public Library of Science ({PLoS})},
    volume = {11},
    number = {10},
    pages = {1004568},
    author = {Sean L. Seyler and Avishek Kumar and M. F. Thorpe and Oliver Beckstein},
    editor = {Emad Tajkhorshid},
    title = {Path Similarity Analysis: A Method for Quantifying Macromolecular Pathways},
    journal = {{PLOS} Computational Biology}
}
drew-parsons commented 4 years ago

Those three references are still making a problem with fresh source (i.e. with no .duecredit.p present before tests).

Today's citation list (still with duecredit 0.7.0) is

[1] Gowers, R. et al., 2016. MDAnalysis: A Python Package for the Rapid Analysis of Molecular Dynamics Simulations. In Proceedings of the 15th Python in Science Conference.  SciPy.
[2] Michaud-Agrawal, N. et al., 2011. MDAnalysis: A toolkit for the analysis of molecular dynamics simulations. Journal of Computational Chemistry, 32(10), pp.2319–2327.
[3] ERRORED: not enough values to unpack (expected 2, got 1)
[4] Gowers, R.J. & Carbone, P., 2015. A multiscale approach to model hydrogen bonding: The case of polyamide. The Journal of Chemical Physics, 142(22), p.224907.
[5] Smart, O.S., Goodfellow, J.M. & Wallace, B.A., 1993. The pore dimensions of gramicidin A. Biophysical Journal, 65(6), pp.2455–2460.
[6] Smart, O.S. et al., 1996. HOLE: A program for the analysis of the pore dimensions of ion channel structural models. Journal of Molecular Graphics, 14(6), pp.354–360.
[7] Stelzl, L.S. et al., 2014. Flexible Gates Generate Occluded Intermediates in the Transport Cycle of LacY. Journal of Molecular Biology, 426(3), pp.735–751.
[8] Smith, P. et al., 2019. On the interaction of hyaluronic acid with synovial fluid lipid membranes. Physical Chemistry Chemical Physics, 21(19), pp.9845–9857.
[9] ERRORED: not enough values to unpack (expected 2, got 1)
[10] Theobald, D.L., 2005. Rapid calculation of RMSDs using a quaternion-based characteristic polynomial. Acta Crystallographica Section A Foundations of Crystallography, 61(4), pp.478–480.
[11] ERRORED: not enough values to unpack (expected 2, got 1)
[12] Van Der Walt, S., Colbert, S.C. & Varoquaux, G., 2011. The NumPy array: a structure for efficient numerical computation. Computing in Science & Engineering, 13(2), pp.22–30.
yarikoptic commented 4 years ago

Sorry for troubles with duecredit. value.replace(' ', '').split('--') issue I believe is due to citeproc, I believe it was fixed since awhile. citeproc in debian is abit outdated, I am looking at updating it now to 0.5.1 now that rcn2rng is there as well. Meanwhile if you could test that upgrade of duecredit + citeproc to the latests in pypi would help -- that would be great and appreciated.

drew-parsons commented 4 years ago

Citations are now clean with duecredit 0.8.0 (aided and abetted by citeproc 0.5.1). Thanks @orbeckst, thanks @yarikoptic

I'll close this issue then.

drew-parsons commented 4 years ago

p.s. pytest-3 -k test_data.py is now running citations cleanly, but a full run with pytest (via Debian's autopkgtest -B -- null) gives one more glitch:

...
[9] Yang, L. et al., 2008. Close Correspondence between the Motions from Principal Component Analysis of Multiple HIV-1 Protease Structures and Elastic Network Modes. Structure, 16(2), pp.321–330.
[10] Seyler, S.L. et al., 2015. Path Similarity Analysis: A Method for Quantifying Macromolecular Pathways E. Tajkhorshid, ed.. PLOS Computational Biology, 11(10), p.1004568.
[11] ERRORED: generator raised StopIteration
[12] Theobald, D.L., 2005. Rapid calculation of RMSDs using a quaternion-based characteristic polynomial. Acta Crystallographica Section A Foundations of Crystallography, 61(4), pp.478–480.
[13] Liu, P., Agrafiotis, D.K. & Theobald, D.L., Fast determination of the optimal rotational matrix for macromolecular superpositions. Journal of Computational Chemistry, 31(7), pp.1561–1563.
[14] Van Der Walt, S., Colbert, S.C. & Varoquaux, G., 2011. The NumPy array: a structure for efficient numerical computation. Computing in Science & Engineering, 13(2), pp.22–30.
Exception ignored in: <function ReaderBase.__del__ at 0x7f80484ee550>
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/MDAnalysis/coordinates/base.py", line 2152, in __del__
    self.close()
  File "/usr/lib/python3/dist-packages/MDAnalysis/coordinates/PDB.py", line 462, in close
    self._pdbfile.close()
  File "/usr/lib/python3/dist-packages/MDAnalysis/lib/util.py", line 739, in close
    self.flush()
  File "/usr/lib/python3/dist-packages/MDAnalysis/lib/util.py", line 845, in flush
    return self.stream.flush()
ValueError: I/O operation on closed file.

The short run on a single test only generates 11 citations, not the 14 cited here. Must be some other test bringing the algorithm cited in this broken [11]. Possibly not worth reopening this bug for it right now, but something to keep an eye on.

I'll file a bug once the mdanalysis package is accepted into the Debian archives, if it's still happening then.

orbeckst commented 4 years ago

It's possible that the test does not collect all citations. E.g., the ENCORE citations are only collected when MDAnalysis.analysis.encore is imported (IIRC).

The ValueError: I/O operation on closed file. is harmless (but admittedly not pretty).