Error when installing 3.0.2 from pypi

[jbarnoud]

Pymbar cannot be installed from pypi because pymbar/_pymbar.c is missing from the tarball.

I try to install pymbar on python 2.7 in a fresh virtualenv:

mkvirtualenv pymbar-test -p python2.7
pip install numpy six
pip install pymbar

Pip tries to install pymbar in version 3.0.2 but fails with the following error message:

Failed to build pymbar
Installing collected packages: pymbar
  Running setup.py install for pymbar ... error
    Complete output from command /coarse/jon/Envs/pymbar/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-KRHJLH/pymbar/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-K8SheN-record/install-record.txt --single-version-externally-managed --compile --install-headers /coarse/jon/Envs/pymbar/include/site/python2.7/pymbar:
                                      author     Levi N. Naden and Michael R. Shirts and John D. Chodera
                                author_emaillevi.naden@choderalab.org, michael.shirts@virginia.edu, john.chodera@choderalab.org
                                  data_files                                                          []
                                 descriptionPython implementation of the multistate Bennett acceptance ratio (MBAR) method.
                                download_url                         http://github.com/choderalab/pymbar
                                 ext_modules[<setuptools.extension.Extension instance at 0x7fad744a5440>]
                                     license                                                         MIT
                            long_description
        Pymbar (https://simtk.org/home/pymbar) is a library
        that provides tools for optimally combining simulations
        from multiple thermodynamic states using maximum likelihood
        methods to compute free energies (normalization constants)
        and expectation values from all of the samples simultaneously.

                                        name                                                      pymbar
                                 package_dir        {'pymbar.tests': 'pymbar/tests', 'pymbar': 'pymbar'}
                                    packages            ['pymbar', 'pymbar.testsystems', 'pymbar.tests']
                                   platforms                            ['Linux', 'Mac OS X', 'Windows']
                                    requires                       ['numpy', 'scipy', 'nose', 'numexpr']
                                         url                         http://github.com/choderalab/pymbar
                                     version                                                       3.0.2
                                    zip_safe                                                       False

    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-2.7
    creating build/lib.linux-x86_64-2.7/pymbar
    copying pymbar/confidenceintervals.py -> build/lib.linux-x86_64-2.7/pymbar
    copying pymbar/bar.py -> build/lib.linux-x86_64-2.7/pymbar
    copying pymbar/exp.py -> build/lib.linux-x86_64-2.7/pymbar
    copying pymbar/utils_for_testing.py -> build/lib.linux-x86_64-2.7/pymbar
    copying pymbar/mbar.py -> build/lib.linux-x86_64-2.7/pymbar
    copying pymbar/timeseries.py -> build/lib.linux-x86_64-2.7/pymbar
    copying pymbar/__init__.py -> build/lib.linux-x86_64-2.7/pymbar
    copying pymbar/mbar_solvers.py -> build/lib.linux-x86_64-2.7/pymbar
    copying pymbar/version.py -> build/lib.linux-x86_64-2.7/pymbar
    copying pymbar/old_mbar.py -> build/lib.linux-x86_64-2.7/pymbar
    copying pymbar/utils.py -> build/lib.linux-x86_64-2.7/pymbar
    creating build/lib.linux-x86_64-2.7/pymbar/testsystems
    copying pymbar/testsystems/exponential_distributions.py -> build/lib.linux-x86_64-2.7/pymbar/testsystems
    copying pymbar/testsystems/harmonic_oscillators.py -> build/lib.linux-x86_64-2.7/pymbar/testsystems
    copying pymbar/testsystems/timeseries.py -> build/lib.linux-x86_64-2.7/pymbar/testsystems
    copying pymbar/testsystems/__init__.py -> build/lib.linux-x86_64-2.7/pymbar/testsystems
    copying pymbar/testsystems/gaussian_work.py -> build/lib.linux-x86_64-2.7/pymbar/testsystems
    creating build/lib.linux-x86_64-2.7/pymbar/tests
    copying pymbar/tests/test_utils.py -> build/lib.linux-x86_64-2.7/pymbar/tests
    copying pymbar/tests/test_covariance.py -> build/lib.linux-x86_64-2.7/pymbar/tests
    copying pymbar/tests/__init__.py -> build/lib.linux-x86_64-2.7/pymbar/tests
    copying pymbar/tests/test_timeseries.py -> build/lib.linux-x86_64-2.7/pymbar/tests
    copying pymbar/tests/test_mbar.py -> build/lib.linux-x86_64-2.7/pymbar/tests
    copying pymbar/tests/test_mbar_solvers.py -> build/lib.linux-x86_64-2.7/pymbar/tests
    running build_ext
    building '_pymbar' extension
    creating build/temp.linux-x86_64-2.7
    creating build/temp.linux-x86_64-2.7/pymbar
    x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/coarse/jon/Envs/pymbar/local/lib/python2.7/site-packages/numpy/core/include -I/coarse/jon/Envs/pymbar/local/lib/python2.7/site-packages/numpy/core/include/numpy/ -I/usr/include/python2.7 -c pymbar/_pymbar.c -o build/temp.linux-x86_64-2.7/pymbar/_pymbar.o -std=c99 -O2 -shared -msse2 -msse3
    x86_64-linux-gnu-gcc: error: pymbar/_pymbar.c: No such file or directory
    x86_64-linux-gnu-gcc: fatal error: no input files
    compilation terminated.
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

    ----------------------------------------
Command "/coarse/jon/Envs/pymbar/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-KRHJLH/pymbar/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-K8SheN-record/install-record.txt --single-version-externally-managed --compile --install-headers /coarse/jon/Envs/pymbar/include/site/python2.7/pymbar" failed with error code 1 in /tmp/pip-build-KRHJLH/pymbar/

The relevant part seems to be:

x86_64-linux-gnu-gcc: error: pymbar/_pymbar.c: No such file or directory

While the file is present in the git repository, it is not in the tarball on pypi:

$ wget https://files.pythonhosted.org/packages/ce/03/937b32fa74aa410bb6d10c4fa7ad622552a24b0557a35129225f6dd2d3da/pymbar-3.0.2.tar.gz
--2017-07-31 14:59:50--  https://files.pythonhosted.org/packages/ce/03/937b32fa74aa410bb6d10c4fa7ad622552a24b0557a35129225f6dd2d3da/pymbar-3.0.2.tar.gz
Resolving files.pythonhosted.org (files.pythonhosted.org)... 151.101.37.63, 2a04:4e42:9::319
Connecting to files.pythonhosted.org (files.pythonhosted.org)|151.101.37.63|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 80421 (79K) [binary/octet-stream]
Saving to: ‘pymbar-3.0.2.tar.gz’

pymbar-3.0.2.tar.gz           100%[===============================================>]  78.54K  --.-KB/s    in 0.01s   

2017-07-31 14:59:50 (6.32 MB/s) - ‘pymbar-3.0.2.tar.gz’ saved [80421/80421]
$ tar -tf pymbar-3.0.2.tar.gz | grep -c _pymbar.c
0

The issue appears to be fixed in the current state of the git repo:

$ git clone https://github.com/choderalab/pymbar.git
$ cd pymbar
$ python setup.py sdist --formats=gztar
$ cd dist
$ tar -tf pymbar-3.0.3.tar.gz | grep -c _pymbar.c
1
$ pip install pymbar-3.0.3.tar.gz 
Processing ./pymbar-3.0.3.tar.gz
Building wheels for collected packages: pymbar
  Running setup.py bdist_wheel for pymbar ... done
  Stored in directory: /home/jon/.cache/pip/wheels/21/73/c7/f525f7211348e57b4aa37b533c12b59e4464cb7d367178aa99
Successfully built pymbar
Installing collected packages: pymbar
Successfully installed pymbar-3.0.3

However, the package that is currently available on pypi does not install.

[gasteigerjo]

I also couldn't install pymbar3.0.2 via pip install on several machines and got the same message as jbarnoud. However, using the current state of the repo (as jbarnoud suggested) fixed this.

Please fix the tarball.

[jchodera]

@Lnaden should be able to tackle it this week!

[Lnaden]

I just cut the release for pymbar 3.0.3 which fixes this issue. There are no changes to the functional code for PyMBAR from 3.0.2, but it did require me to change the distribution code a bit, hence the version bump.

Let us know if there are any further issues!

Long version of what happened in case you want to know, and for documentation:

The _pymbar.c extension is configured only to compile on Python 2.X. If the tarball was created with Python 3.X, the extension is never compiled so the .c file is ignored by default and not included with the tarball. This is what happened with 3.0.2 and also why installing from source works, because the .c file is present in the GitHub source.