py.test error

[mbgnl]

Hi,

I have installed pyGNOME via Anaconda as suggested at https://gnome.orr.noaa.gov/doc/pygnome/installing.html, but when I try to run py.test from ~/PyGnome/py_gnome/tests/unit_tests I get the following errors:

ImportError while loading conftest '/home/PyGnome/py_gnome/tests/conftest.py'. ../conftest.py:15: in from gnome.utilities import rand ../../../../anaconda2/envs/gnome/lib/python2.7/site-packages/pyGnome-0.6.5-py2.7-linux-x86_64.egg/gnome/init.py:116: in from . import (environment, ../../../../anaconda2/envs/gnome/lib/python2.7/site-packages/pyGnome-0.6.5-py2.7-linux-x86_64.egg/gnome/model.py:18: in from gnome.utilities.cache import ElementCache ../../../../anaconda2/envs/gnome/lib/python2.7/site-packages/pyGnome-0.6.5-py2.7-linux-x86_64.egg/gnome/utilities/cache.py:18: in from gnome.spill_container import (SpillContainerData, ../../../../anaconda2/envs/gnome/lib/python2.7/site-packages/pyGnome-0.6.5-py2.7-linux-x86_64.egg/gnome/spill_container.py:21: in import gnome.spill ../../../../anaconda2/envs/gnome/lib/python2.7/site-packages/pyGnome-0.6.5-py2.7-linux-x86_64.egg/gnome/spill/init.py:1: in from spill import (Spill, ../../../../anaconda2/envs/gnome/lib/python2.7/site-packages/pyGnome-0.6.5-py2.7-linux-x86_64.egg/gnome/spill/spill.py:26: in from .release import (Release, ../../../../anaconda2/envs/gnome/lib/python2.7/site-packages/pyGnome-0.6.5-py2.7-linux-x86_64.egg/gnome/spill/release.py:36: in from gnome.outputters import NetCDFOutput ../../../../anaconda2/envs/gnome/lib/python2.7/site-packages/pyGnome-0.6.5-py2.7-linux-x86_64.egg/gnome/outputters/init.py:4: in from renderer import Renderer, RendererSchema ../../../../anaconda2/envs/gnome/lib/python2.7/site-packages/pyGnome-0.6.5-py2.7-linux-x86_64.egg/gnome/outputters/renderer.py:14: in import py_gd ../../../../anaconda2/envs/gnome/lib/python2.7/site-packages/py_gd/init.py:40: in from py_gd import * E ImportError: libjpeg.so.8: cannot open shared object file: No such file or directory

As for this last error concerning libjpeg.so.8, I have libjpeg.so.62 in my system, but either py.test does not find it or there is a compatibility issue.

How can I possibly fix the above errors? My OS is CentOS Linux 7 (Core).

Thank you in advance for any help.

Max

[mbgnl]

The above problem has been solved by installing libjpeg-turbo via conda.

Unfortunately, now I get the following error message while running py.test:

test_maps/test_tideflat_map.py ..... Error in `~/anaconda2/envs/gnome/bin/python': free(): invalid pointer: 0x00002b2611f47000

Any idea on how to solve it?

Thank you in advance,

Max

[ChrisBarker-NOAA]

"The above problem has been solved by installing libjpeg-turbo via conda."

Hmm -- that seems to be an issue with the conda recipe for libgd then. If you have the energy, an issue on:

https://github.com/conda-forge/libgd-feedstock

be sure to put what version of libgd was installed:

$ conda list libgd
# packages in environment at /Users/chris.barker/miniconda3/envs/gnome:
#
# Name                    Version                   Build  Channel
libgd                     2.3.0                h55673e3_1    conda-forge

[ChrisBarker-NOAA]

"The above problem has been solved by installing libjpeg-turbo via conda."

Hmm -- that seems to be an issue with the conda recipe for libgd then. If you have the energy, an issue on:

https://github.com/conda-forge/libgd-feedstock

be sure to put what version of libgd was installed:

$ conda list libgd
# packages in environment at /Users/chris.barker/miniconda3/envs/gnome:
#
# Name                    Version                   Build  Channel
libgd                     2.3.0                h55673e3_1    conda-forge

[ChrisBarker-NOAA]

Unfortunately, now I get the following error message while running py.test:

test_maps/test_tideflat_map.py ..... Error in `~/anaconda2/envs/gnome/bin/python': free(): invalid pointer: 0x00002b2611f47000

Wow! that is a really odd one. I can suggest a few things:

1st step -- we have just made a transition to Python 3.8 -- it's not fully tested, so not in the master branch yet, but that's where all future development is headed. So working with the "develop" branch, and Python3.8 may be a better way to go forward.

In either case:

1) first, clean everything out and rebuild:

python setup.py cleanall

python setup.py develop

then try the tests:

cd tests/unit_tests

pytest

If the problem persists, the try running the tests in smaller batches -- first with the one that's crashing:

cd test_maps
pytesttest_tideflat_map.py

Then maybe each test dir at a time, and see if we can nai down what code is failing. I can't imagine that the problem is with the tide flat map, as there is no C++ code involved specifically with that, but it's a place to start.

It is possible that the issue you had with ligd is related -- the map code used libgd, via py_gd -- so the error could come from there.

Testing lib_gd

It might be a good idea to do an isolated test of libgd in a clean environment:

$ conda create -n gd_test python=3.8

$ conda activate gd_test

$ conda install py_gd

$ conda install py_gd pytest

$ pytest --pyargs py_gd
============================= test session starts ==============================
platform darwin -- Python 3.8.8, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: /Users/chris.barker
collected 101 items                                                            

miniconda3/envs/gd_test/lib/python3.8/site-packages/py_gd/test/test_buffer.py s [  0%]
                                                                         [  0%]
miniconda3/envs/gd_test/lib/python3.8/site-packages/py_gd/test/test_colors.py . [  1%]
.....................                                                    [ 22%]
miniconda3/envs/gd_test/lib/python3.8/site-packages/py_gd/test/test_gd.py . [ 23%]
...........................................................              [ 82%]
miniconda3/envs/gd_test/lib/python3.8/site-packages/py_gd/test/test_overflow.py . [ 83%]
.................                                                        [100%]

======================== 100 passed, 1 skipped in 1.51s ========================
(gd_test) ORR-M-4420442:~ chris.barker$ 

[mbgnl]

Dear Mr. Barker,

Thank you very much for your quick response.

Following your suggestion, I cleaned up both my Anaconda and pyGNOME installation and I reinstalled them from scratch with Python 3.8. Now py.test correctly works, and the test summary info I get is the following:

1204 passed, 93 skipped, 29 xfailed, 2 xpassed, 4787 warnings in 191.58s (0:03:11)

No libgd errors occured with such installation.

However, I get 2 failed tests when I run py.test --runslow:

FAILED test_cy/test_cy_grid_map.py::TestGridMap::test_grid_map_curv - TypeError: expected bytes, str found FAILED test_cy/test_cy_grid_map.py::TestGridMap::test_grid_map_tri - TypeError: expected bytes, str found 2 failed, 1247 passed, 48 skipped, 29 xfailed, 2 xpassed, 6746 warnings in 282.61s (0:04:42)

Do you know where those errors come from? Also, are my test results sufficiently good to start working with pyGNOME or should I consider rebuilding it first?

As for my previous libgd issue, I wanted to open an issue at https://github.com/conda-forge/libgd-feedstock, but then I realized I had already cleaned up my conda installation and reinstalled the Anaconda version for Python 3.8, so I could not retrieve my previous libgd version. Sorry for that.

Thanks again for your help.

Max

[ChrisBarker-NOAA]

Regarding:

However, I get 2 failed tests when I run py.test --runslow:

These are known erros -- this IS the develop banch :-)

But you're in fine shape to proceed, those are rarely used features that should be fixed soon.

As for libgd -- if you don't have problem siwth a clean install, it's not a big deal.