horton-2.1.0 does not install on latest fedora 26 & 27

[maaren]

I followed: http://theochem.github.io/horton/2.1.0/user_download_and_install_linux.html all went well untill:

$ ./setup.py install --user > install.log 2>&1
...
cythoning horton/grid/cext.pyx to horton/grid/cext.cpp

Error compiling Cython file:
------------------------------------------------------------
...
# evaluate
#

def index_wrap(long i, long high):
    return evaluate.index_wrap(i, high)
                  ^
------------------------------------------------------------
...

I have attached the full output for the fedora 26 system. On my fedora 27 system it breaks in the same location. Please let me know if you need more info.

install.log

To anwser my own question, removing the following index_wrap references from horton-2.1.0/horton/grid/cext.pyx seems to work:

$ diff horton-2.1.0/horton/grid/cext.pyx horton-2.1.0.org/horton/grid/cext.pyx 
58c58
<     'eval_spline_grid', 'eval_decomposition_grid',
---
>     'index_wrap', 'eval_spline_grid', 'eval_decomposition_grid',
477a478,481
> 
> 
> def index_wrap(long i, long high):
>     return evaluate.index_wrap(i, high)

[matt-chan]

Yes, this is a known issue. Sorry I didn't get around to filling it earlier.

The version of cython in the fedora repo (0.26) breaks the code. The short-term workaround is to use an earlier version. There is a version of Horton in the conda theochem channel that has all the dependencies already built and compiling successfully.

We have a new version of Horton coming out which will fix this issue permanently. It's a major rewrite though, so it's not going to be a matter of a week or two. If you absolutely need the functionality through the fedora repos soon, please let me know and I can try to push a fix for 2.1.0. On Sun, Dec 3, 2017 at 10:59 AM maaren notifications@github.com wrote:

I followed: http://theochem.github.io/horton/2.1.0/user_download_and_install_linux.html all went well untill:

$ ./setup.py install --user > install.log 2>&1 ... cythoning horton/grid/cext.pyx to horton/grid/cext.cpp Error compiling Cython file:

... evaluate def index_wrap(long i, long high): return evaluate.index_wrap(i, high) ^

...

I have attached the full output for the fedora 26 system. On my fedora 27 system it breaks in the same location. install.log https://github.com/theochem/horton/files/1524920/install.log

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/theochem/horton/issues/282, or mute the thread https://github.com/notifications/unsubscribe-auth/AA_-NRf6oAjmrxku0poGLcHttNjeF65Yks5s8sVdgaJpZM4QzxWj .

-- Matt

Sent from my phone

[maaren]

by hacking: horton-2.1.0/horton/grid/cext.pyx I managed to get it compiled and installed, but I get quite a number of errors when I run nosetests... I will have a look at the conda version.

[tovrstra]

The fix is already available in the master branch. See #268

[matt-chan]

Ah oops. Maybe we should make another release. 2.1.1?

On Mon, 4 Dec 2017 at 14:47 Toon Verstraelen notifications@github.com wrote:

The fix is already available in the master branch. See #268 https://github.com/theochem/horton/pull/268

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/theochem/horton/issues/282#issuecomment-349083465, or mute the thread https://github.com/notifications/unsubscribe-auth/AA_-NcFK3_hdjnTlvz9GGjGoTNmY2P4qks5s9ExigaJpZM4QzxWj .

-- Matt

Sent from my phone

[tovrstra]

Fine by me but I'd suggest to make a maintenance branch rather releasing the current master as 2.1.1 (many backward incompatible changes). It may be worth skimming through the commits to look for other small fixes and improvements.

[maaren]

I managed to compile and install the master branch, but when I run nosetests -v horton I have 110 errors. Is that to be expected?

[tovrstra]

No, but keep in mind that the master branch contains some rather severe backward-incompatible changes w.r.t. to the 2.1.0 release. These were mostly experimental changes we did to explore our future options. Normally the tests should run, but you need to link to libxc3.

[maaren]

I think it was picking up stuff from previous installs, after cleaning everything I am left with:

horton.meanfield.test.test_libxc.test_dot_hessian_o3lyp_cs_polynomial ... SKIP: We should use more robust tests for derivatives.
horton.part.test.test_proatomdb.test_db_basics ... FAIL
horton.part.test.test_proatomdb.test_io_group ... FAIL
horton.part.test.test_proatomdb.test_io_filename ... FAIL
horton.part.test.test_proatomdb.test_compute_radii ... ERROR
horton.part.test.test_proatomdb.test_get_spline ... ERROR
horton.scripts.test.test_atomdb.test_plot_atoms ... FAIL
FAILED (SKIP=1, errors=2, failures=4)

[tovrstra]

Can you also paste the details of the test fails and errors?

[maaren]

======================================================================
ERROR: horton.part.test.test_proatomdb.test_compute_radii
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/maaren/.local/lib/python2.7/site-packages/horton/part/test/test_pr
oatomdb.py", line 128, in test_compute_radii
    record = padb.get_record(6, 0)
  File "/home/maaren/.local/lib/python2.7/site-packages/horton/part/proatomdb.py
", line 358, in get_record
    return self._map[(number, charge)]
KeyError: (6, 0)

======================================================================
ERROR: horton.part.test.test_proatomdb.test_get_spline
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/maaren/.local/lib/python2.7/site-packages/horton/part/test/test_pr
oatomdb.py", line 172, in test_get_spline
    spline = padb.get_spline(6)
  File "/home/maaren/.local/lib/python2.7/site-packages/horton/part/proatomdb.py
", line 597, in get_spline
    rho, deriv = self.get_rho(number, parameters, combine, do_deriv=True)
  File "/home/maaren/.local/lib/python2.7/site-packages/horton/part/proatomdb.py
", line 547, in get_rho
    record = self.get_record(number, charge)
  File "/home/maaren/.local/lib/python2.7/site-packages/horton/part/proatomdb.py", line 358, in get_record
    return self._map[(number, charge)]
KeyError: (6, 0)

======================================================================
FAIL: horton.part.test.test_proatomdb.test_db_basics
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/maaren/.local/lib/python2.7/site-packages/horton/part/test/test_proatomdb.py", line 32, in test_db_basics
    assert padb.get_numbers() == [1, 8]
AssertionError

======================================================================
FAIL: horton.part.test.test_proatomdb.test_io_group
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/maaren/.local/lib/python2.7/site-packages/horton/part/test/test_proatomdb.py", line 103, in test_io_group
    assert padb1.size == 5
AssertionError

======================================================================
FAIL: horton.part.test.test_proatomdb.test_io_filename
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/maaren/.local/lib/python2.7/site-packages/horton/part/test/test_proatomdb.py", line 116, in test_io_filename
    assert keys == [(1, 0), (6, 0), (6, 1)]
AssertionError

======================================================================
FAIL: horton.scripts.test.test_atomdb.test_plot_atoms
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/maaren/.local/lib/python2.7/site-packages/horton/scripts/test/test_atomdb.py", line 209, in test_plot_atoms
    check_files(dn, fns)
  File "/home/maaren/.local/lib/python2.7/site-packages/horton/scripts/test/common.py", line 39, in check_files
    assert os.path.isfile(os.path.join(dn, fn)), "Missing %s" % fn
AssertionError: Missing dens_001__h.png

----------------------------------------------------------------------

[tovrstra]

OK, that should be easy to fix. Can you try the following: http://theochem.github.io/horton/2.1.0/tech_dev_git.html#reference-atoms

[maaren]

Thanks Toon! Apart from one SKIP, everything is ok now.

[tovrstra]

Great. Be careful with the master branch because it contains a lot of experimental changes, some of which are definitely for the better. E.g. we stripped a lot of redundant infrastructure code relative to the 2.1.0 release. If you build the documentation from the master branch, it should also be in sync with the changes in the code.

[maaren]

OK, I will try to build the documentation too.

I was wondering if install_libxc-3.0.0.sh and install_libint-2.0.3.sh are still required on fedora f26/f27? The libs libint2-2.1.0 and libxc-3.0.1 are available in the OS repo.

[tovrstra]

I'm not sure if that will work for LibInt. For LibXC, this should just work.

[maaren]

I tried it on my f27 system, and it seems to work:

$ ldd ~/.local/lib/python2.7/site-packages/horton/gbasis/cext.so | grep libint2
        libint2-beta.so.2 => /lib64/libint2-beta.so.2 (0x00007fcad1900000)
$ ldd ~/.local/lib/python2.7/site-packages/horton/meanfield/cext.so | grep libxc
        libxc.so.4 => /lib64/libxc.so.4 (0x00007f0ad1f3a000)
$ nosetests -v horton 
...
OK (SKIP=1)

[tovrstra]

Nice. Thanks for letting us know.

[maaren]

I assume you are aware, but there is a hard-coded dependency on qaworkdir/cached/libxc-x.x.x to build the documentation. I think there are also some javascript files missing under doc/_static (I downloaded doctools.js, jquery.js & underscore.js manually to get the equations to render)

[tovrstra]

Yes, there is no other way to extrac the list of functionals from LibXC. (There is no API to get the full list from a compiled instance of the library, as far as I know.)

It should normally not be needed to include these javascripts. At least, I haven't noticed the need yet. I just double checked on my machine and documentation builds without adding these scripts. Could it be one of the dependencies that is missing? There are additional dependencies for building the documentation. See http://theochem.github.io/horton/2.1.0/tech_dev_git.html#building-the-documentation

[maaren]

I am doing this on f27, and the dependencies are there:

Package python2-sphinx-1:1.6.5-1.fc27.noarch is already installed, skipping.
Package python2-sphinx_rtd_theme-0.2.4-2.fc27.noarch is already installed, skipping.
Package doxygen-1:1.8.13-10.fc27.x86_64 is already installed, skipping.
Package python2-breathe-4.7.3-1.fc27.noarch is already installed, skipping.
Package python2-docutils-0.13.1-6.fc27.noarch is already installed, skipping.

Everything seems to be generated OK, only the _static directory only contained custom.css, and the equations did not render. I checked the HTML code and noticed:

$ grep _static _build/html/user_postproc_espfit.html 
    <link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
    <link rel="stylesheet" href="_static/custom.css" type="text/css" />
  <script src="_static/js/modernizr.min.js"></script>
            <img src="_static/horton.png" class="logo" />
      <script type="text/javascript" src="_static/jquery.js"></script>
      <script type="text/javascript" src="_static/underscore.js"></script>
      <script type="text/javascript" src="_static/doctools.js"></script>
    <script type="text/javascript" src="_static/js/theme.js"></script>

After I downloaded:

http://theochem.github.io/horton/2.1.0/_static/jquery.js
http://theochem.github.io/horton/2.1.0/_static/underscore.js
http://theochem.github.io/horton/2.1.0/_static/doctools.js

the equations rendered. I have not tried them individually to figure out if all are required. modernizr.min.js and js/theme.js don't seem to be necessary.

[matt-chan]

We were missing them from the MANIFEST.in.

I've fixed it in master. Sorry about that! Matt

On Tue, 12 Dec 2017 at 00:55 maaren notifications@github.com wrote:

I am doing this on f27, and the dependencies are there:

Package python2-sphinx-1:1.6.5-1.fc27.noarch is already installed, skipping. Package python2-sphinx_rtd_theme-0.2.4-2.fc27.noarch is already installed, skipping. Package doxygen-1:1.8.13-10.fc27.x86_64 is already installed, skipping. Package python2-breathe-4.7.3-1.fc27.noarch is already installed, skipping. Package python2-docutils-0.13.1-6.fc27.noarch is already installed, skipping.

Everything seems to be generated OK, only the _static directory only contained custom.css, and the equations did not render. I checked the HTML code and noticed:

$ grep _static _build/html/user_postproc_espfit.html

<link rel="stylesheet" href="_static/custom.css" type="text/css" />

        <img src="_static/horton.png" class="logo" />
  <script type="text/javascript" src="_static/jquery.js"></script>
  <script type="text/javascript" src="_static/underscore.js"></script>
  <script type="text/javascript" src="_static/doctools.js"></script>
<script type="text/javascript" src="_static/js/theme.js"></script>

After I downloaded them:

http://theochem.github.io/horton/2.1.0/_static/jquery.jshttp://theochem.github.io/horton/2.1.0/_static/underscore.jshttp://theochem.github.io/horton/2.1.0/_static/doctools.js

it started to work.

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/theochem/horton/issues/282#issuecomment-350953112, or mute the thread https://github.com/notifications/unsubscribe-auth/AA_-NT76MnEYCrEo3taPUQrSb_IcXpC3ks5s_hVRgaJpZM4QzxWj .

-- Matt

Sent from my phone

[maaren]

Are you sure? After make clean I get:

$ make html > make-html.log 2>&1
$ ls _static/
custom.css

[tovrstra]

I don't have the javascripts in my source tree under doc/_static and still the documentation works, with equations rendered with mathjax. These files do not need to be present in the source tree. Instead they are added automatically in doc/_build/html/_static when you compile the docs.

@matt-chan: can you remove them again from MANIFEST.in?

@maaren: can you copy the output of the setup.py script with the list of dependencies?

This is what I have:

Version of dependencies:
             breathe: 4.7.3
            coverage: /bin/sh: coverage: command not found
            cppcheck: Cppcheck 1.80
                curl: curl 7.55.1 (x86_64-redhat-linux-gnu) libcurl/7.55.1 OpenSSL/1.1.0g zlib/1.2.11 libidn2/2.0.4 libpsl/0.18.0 (+libidn2/2.0.3) libssh2/1.8.0 nghttp2/1.25.0
              cython: Cython version 0.26.1
            docutils: 0.13.1
             doxygen: 1.8.13
                 gcc: gcc (GCC) 7.2.1 20170915 (Red Hat 7.2.1-2)
                 git: git version 2.14.3
           gitpython: -- not found --
                h5py: 2.7.1
          matplotlib: 2.0.0
                nose: nosetests version 1.3.7
               numpy: 1.13.1
                 pip: pip 9.0.1 from /usr/lib/python2.7/site-packages (python 2.7)
         pycodestyle: 2.3.1
          pydocstyle: 2.0.0
              pylint: pylint 1.7.2,     astroid 1.5.3
              python: Python 2.7.14
               scipy: 0.19.1
              sphinx: Sphinx (sphinx-build) 1.6.5
    sphinx_rtd_theme: 0.2.4
               sympy: 1.1.1

Especially the sphinx version could be relevant.

[maaren]

Mmm, I am not sure what I have done, but it seems to work now...

I had the issue on f27. So I tried it on my f26 system and it worked fine. So I tried again on the f27 system, and everything is rendering now. These are the dependencies on f26:

Version of dependencies:
             breathe: 3.1.0
            coverage: Coverage.py, version 4.4.1 with C extension
            cppcheck: -- not found --
                curl: curl 7.53.1 (x86_64-redhat-linux-gnu) libcurl/7.53.1 NSS/3.33 zlib/1.2.11 libidn2/2.0.4 libpsl/0.18.0 (+libidn2/2.0.3) libssh2/1.8.0 nghttp2/1.21.1
              cython: Cython version 0.27.3
            docutils: 0.13.1
             doxygen: 1.8.13
                 gcc: gcc (GCC) 7.2.1 20170915 (Red Hat 7.2.1-2)
                 git: git version 2.13.6
           gitpython: 2.1.7
                h5py: 2.7.1
          matplotlib: 2.0.0
                nose: nosetests version 1.3.7
               numpy: 1.12.1
                 pip: pip 9.0.1 from /usr/lib/python2.7/site-packages (python 2.7)
         pycodestyle: -- not found --
          pydocstyle: -- not found --
              pylint: 
              python: Python 2.7.14
               scipy: 0.19.1
              sphinx: Sphinx (sphinx-build) 1.5.5
    sphinx_rtd_theme: 0.2.4
               sympy: 1.1.1

And these on f27:

Version of dependencies:
             breathe: -- not found --
            coverage: Coverage.py, version 4.4.2 with C extension
            cppcheck: -- not found --
                curl: curl 7.55.1 (x86_64-redhat-linux-gnu) libcurl/7.55.1 OpenSSL/1.1.0g zlib/1.2.11 libidn2/2.0.4 libpsl/0.18.0 (+libidn2/2.0.3) libssh2/1.8.0 nghttp2/1.25.0
              cython: Cython version 0.27.3
            docutils: 0.13.1
             doxygen: 1.8.13
                 gcc: gcc (GCC) 7.2.1 20170915 (Red Hat 7.2.1-2)
                 git: git version 2.14.3
           gitpython: -- not found --
                h5py: 2.7.1
          matplotlib: 2.0.0
                nose: nosetests version 1.3.7
               numpy: 1.13.3
                 pip: pip 9.0.1 from /usr/lib/python2.7/site-packages (python 2.7)
         pycodestyle: -- not found --
          pydocstyle: -- not found --
              pylint:
              python: Python 2.7.14
               scipy: 0.19.1
              sphinx: Sphinx (sphinx-build) 1.6.5
    sphinx_rtd_theme: 0.2.4
               sympy: -- not found --

[tovrstra]

OK. on f27, part of the documentation should look better with the package python-breathe installed. Other than that, I can't really recommend anything. If all works and you have no further questions, feel free to close the issue.

[maaren]

Thanks, everything seems to be working at the moment, so I will close the issue.

[tovrstra]

I just made a bugfix release (horton 2.1.1) that contains the fix for this problem.