search

conda-forge / vtk-feedstock

A conda-smithy repository for vtk.
BSD 3-Clause "New" or "Revised" License
13 stars 64 forks source link

Not finding vtkIOParallelPython in Python3.6 build linux-64 #46

Closed siboles closed 6 years ago

siboles commented 7 years ago

I produced the error below with the vtk=7.1.1=py36_202 build, but 7.0* also had the same issue.

ERROR TRACE

Python 3.6.2 |Intel Corporation| (default, Aug 15 2017, 16:14:26) [GCC 4.8.2 20140120 (Red Hat 4.8.2-15)] on linux Type "help", "copyright", "credits" or "license" for more information. Intel(R) Distribution for Python is brought to you by Intel Corporation. Please check out: https://software.intel.com/en-us/python-distribution

import vtk Traceback (most recent call last): File "/home/scott/anaconda2/envs/vtk3/lib/python3.6/site-packages/vtk/vtkIOParallel.py", line 5, in from .vtkIOParallelPython import * ImportError: libjsoncpp.so.0: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "", line 1, in File "/home/scott/anaconda2/envs/vtk3/lib/python3.6/site-packages/vtk/init.py", line 112, in from .vtkIOParallel import File "/home/scott/anaconda2/envs/vtk3/lib/python3.6/site-packages/vtk/vtkIOParallel.py", line 9, in from vtkIOParallelPython import ModuleNotFoundError: No module named 'vtkIOParallelPython'

ENVIRONMENT

name: vtk3 channels:

OS

Arch Linux 4.9.48-1-lts 64bit

marcelotrevisani commented 7 years ago

Try to install this library libjsoncpp and also install the libjsoncpp-dev package https://www.archlinux.org/packages/extra/x86_64/jsoncpp/

marcelotrevisani commented 7 years ago

You also can install that using conda. The name of the package is jsoncpp. Actually it's kind of weird this dependence...

siboles commented 7 years ago

Thank you for replying. I had the jsoncpp==1.8.3 from defaults conda channel, but this seems to be the problem. Installing version 0.10.6.1from conda-forge resolved the issue. Perhaps, jsoncpp==0.10.6 should be added to meta.yaml.

marcelotrevisani commented 7 years ago

Uunnn... Weird.... I think this problem occurs just in Linux, I will try to add that to the meta.yaml Thanks

ccordoba12 commented 7 years ago

Conda-forge and defaults packages are not binary compatible, so that's most probably the problem here.

This means that if you install packages from conda-forge, every binary package needs to be pulled from conda-forge. The same goes for defaults.

jakirkham commented 7 years ago

@ccordoba12 is correct. We’ve been trying to improve this, but it is a bit of a moving target.

Though would add to that the channel ordering is wrong here. conda-forge must be higher than defaults in priority. Can’t speak to the rest of the channels. Though that one ordering issue alone will cause a lot of grief as it is opposite to how everything is built in conda-forge. Given this, it is totally fair to close this issue and note this is an unsupported configuration.

marcelotrevisani commented 6 years ago

Should we close this issue?

marcelotrevisani commented 6 years ago

I'm going to close this issue.

normanius commented 6 years ago

I read this thread, but as an outsider, I was not able to follow. How can I resolve the problem? Which packages do I have to pull from conda-forge?

I have the same issue on MacOS, conda 4.3.34. I pulled vtk-8.1.0-py36, and end up having the same ImportError when importing vtk.

Thanks!

marcelotrevisani commented 6 years ago

@normanius as you can see on the main page we disabled the build for MacOS

normanius commented 6 years ago

Oh, bummer. Will issue #47 be fixed? Any estimates on the timeline?

In any case thanks a lot for all the efforts. vtk from conda-forge (with Qt support!) saved me for Win. I owe you already something :)

marcelotrevisani commented 6 years ago

@normanius , The problem for me is: I can't test that because I don't have a machine with MacOS. But they tried to fix that for MacOS but as you can see in the discussion there are some problems with the CI in this OS. I think they just gave up on that, but if you really need it, you can test on your own machine and try to solve the problem. Unfortunately, I can't help you more than that :(

If you fix that I will be happy to review your PR.

Maybe @msarahan can update you regarding this topic.

normanius commented 6 years ago

Any instructions on how to debug/trace the problem?

alexdiem commented 6 years ago

I've just encountered the vtkIOParallelPython error as well, even after downgrading to jsoncpp=0.10.6-1 from conda-forge. I'm on Linux Mint 18, Python 3.6.4 | Anaconda custom (64-bit) with packages

In [1]: import vtk
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
~/anaconda3/lib/python3.6/site-packages/vtk/vtkIOParallel.py in <module>()
      4     # use relative import for installed modules
----> 5     from .vtkIOParallelPython import *
      6 except ImportError:

ImportError: libjsoncpp.so.19: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-1-7c7f66f02cd3> in <module>()
----> 1 import vtk

~/anaconda3/lib/python3.6/site-packages/vtk/__init__.py in <module>()
     78 from .vtkIONetCDF import *
     79 from .vtkIOPLY import *
---> 80 from .vtkIOParallel import *
     81 from .vtkIOParallelXML import *
     82 from .vtkIOSQL import *

~/anaconda3/lib/python3.6/site-packages/vtk/vtkIOParallel.py in <module>()
      7     # during build and testing, the modules will be elsewhere,
      8     # e.g. in lib directory or Release/Debug config directories
----> 9     from vtkIOParallelPython import *

ModuleNotFoundError: No module named 'vtkIOParallelPython'
siboles commented 6 years ago

@akdiem I think your issue is the same as mine. You must give higher priority to the conda-forge channel than defaults in your .condarc (i.e. place it higher in the list) since as @ccordoba12 pointed out they are not binary compatible. Also, I would recommend installing in a new conda environment instead of your root.

alexdiem commented 6 years ago

@siboles Thanks for you response, I will do that. However, before commenting I did exactly what you did, i.e. specifically install jsoncpp 0.10.6-1 from conda-forge and that did not fix it for me.

siboles commented 6 years ago

@akdiem Yes, but you have other binaries that are being fetched from e.g. the anaconda channel. In fact, your vtk itself is fetched from there.

alexdiem commented 6 years ago

@siboles Ok, so I deleted the environment entirely and set it up completely from scratch with conda-forge having a higher priority than the default channel. I still get the exact same error. This is the output of conda list:

#
# Name                    Version                   Build  Channel
ca-certificates           2018.4.16                     0    conda-forge
certifi                   2018.4.16                py27_0    conda-forge
freetype                  2.8.1                         0    conda-forge
future                    0.16.0                     py_1    conda-forge
h5py                      2.7.1                    py27_1    conda-forge
hdf5                      1.8.18                        3    conda-forge
intel-openmp              2018.0.3                      0  
itk                       4.13.0               h2f4ac57_1    vmtk
jpeg                      9c                   h470a237_0    conda-forge
jsoncpp                   0.10.6                        1    conda-forge
libgcc-ng                 7.2.0                hdf63c60_3  
libgfortran               3.0.0                         1  
libgfortran-ng            7.2.0                hdf63c60_3  
libpng                    1.6.34               ha92aebf_1    conda-forge
libstdcxx-ng              7.2.0                hdf63c60_3  
libtiff                   4.0.9                he6b73bb_1    conda-forge
llvm                      3.3                           0  
mesa                      10.5.4                        0  
mkl                       2018.0.3                      1  
ncurses                   5.9                          10    conda-forge
numpy                     1.12.1           py27h9378851_1  
olefile                   0.45.1                     py_1    conda-forge
openssl                   1.0.2o                        0    conda-forge
pillow                    5.2.0                    py27_0    conda-forge
pip                       9.0.3                    py27_0    conda-forge
python                    2.7.14                        5    conda-forge
readline                  7.0                           0    conda-forge
setuptools                40.0.0                   py27_0    conda-forge
six                       1.11.0                   py27_1    conda-forge
sqlite                    3.20.1                        2    conda-forge
system                    5.8                           2  
tk                        8.6.7                         0    conda-forge
vmtk                      1.4.0            py27hfe14cac_1    vmtk
vtk                       8.1.0            py27h67b79b7_1    vmtk
wheel                     0.31.1                   py27_0    conda-forge
xz                        5.2.3                         0    conda-forge
zlib                      1.2.11               h470a237_3    conda-forge
siboles commented 6 years ago

@akdiem My guess is the vtk from the vmtk channel is causing the problems. I would try the vtk from conda-forge without installing the vmtk module just for debug purposes. If that is the issue, perhaps then you could force install the conda-forge vtk afterwards and hope that vmtk will work with it.

alexdiem commented 6 years ago

@siboles I created a new conda environment again and as you suggested just installed vtk from conda-forge and made sure that jsoncpp is the correct version. The error persists.

siboles commented 6 years ago

@akdiem Hmm. I can't reproduce the error on my machine running Arch Linux. I created the following environment: conda create -n vtk_error python=2.7 vtk

ca-certificates           2018.4.16                     0    conda-forge
certifi                   2018.4.16                py27_0    conda-forge
curl                      7.60.0                        0    conda-forge
expat                     2.2.5                hfc679d8_1    conda-forge
freetype                  2.8.1                         0    conda-forge
future                    0.16.0                   py27_2    conda-forge
hdf4                      4.2.13                        0    conda-forge
hdf5                      1.10.1                        2    conda-forge
icu                       58.2                 hfc679d8_0    conda-forge
jpeg                      9c                   h470a237_0    conda-forge
jsoncpp                   1.8.1                h2d50403_1    conda-forge
krb5                      1.14.6                        0    conda-forge
libgcc-ng                 7.2.0                hdf63c60_3  
libgfortran               3.0.0                         1  
libiconv                  1.15                 h470a237_1    conda-forge
libnetcdf                 4.6.1                         2    conda-forge
libogg                    1.3.2                         0    conda-forge
libpng                    1.6.34               ha92aebf_1    conda-forge
libssh2                   1.8.0                h5b517e9_2    conda-forge
libstdcxx-ng              7.2.0                hdf63c60_3  
libtheora                 1.1.1                h470a237_1    conda-forge
libtiff                   4.0.9                he6b73bb_1    conda-forge
libuuid                   1.0.3                         1    conda-forge
libvorbis                 1.3.5                         0    conda-forge
libxcb                    1.13                 h470a237_2    conda-forge
libxml2                   2.9.8                h422b904_2    conda-forge
lz4-c                     1.8.1.2                       0    conda-forge
ncurses                   5.9                          10    conda-forge
openssl                   1.0.2o                        0    conda-forge
pip                       9.0.3                    py27_0    conda-forge
pthread-stubs             0.4                  h470a237_1    conda-forge
python                    2.7.15                        0    conda-forge
readline                  7.0                           0    conda-forge
setuptools                40.0.0                   py27_0    conda-forge
sqlite                    3.20.1                        2    conda-forge
tbb                       2018_20171205                 0    conda-forge
tk                        8.6.8                         0    conda-forge
vtk                       8.1.1           py27hdfe3907_203    conda-forge
wheel                     0.31.1                   py27_0    conda-forge
xorg-kbproto              1.0.7                h470a237_2    conda-forge
xorg-libice               1.0.9                h470a237_3    conda-forge
xorg-libsm                1.2.2                h470a237_3    conda-forge
xorg-libx11               1.6.5                h470a237_1    conda-forge
xorg-libxau               1.0.8                h470a237_6    conda-forge
xorg-libxdmcp             1.1.2                h470a237_7    conda-forge
xorg-libxt                1.1.5                h470a237_1    conda-forge
xorg-xproto               7.0.31               h470a237_7    conda-forge
xz                        5.2.3                         0    conda-forge
zlib                      1.2.11               h470a237_3    conda-forge

VTK imports fine for me.

alexdiem commented 6 years ago

@siboles You used Python 2.7 here though and you originally produced the error in 3.6. If I run conda create -n vtk_error python=3.6 vtk I get the import error.

siboles commented 6 years ago

@akdiem I created it with 2.7 because your last post used that. With 3.6 it still works for me. Here is that conda list.

# Name                    Version                   Build  Channel
ca-certificates           2018.4.16                     0    conda-forge
certifi                   2018.4.16                py36_0    conda-forge
curl                      7.60.0                        0    conda-forge
expat                     2.2.5                hfc679d8_1    conda-forge
freetype                  2.8.1                         0    conda-forge
future                    0.16.0                   py36_2    conda-forge
hdf4                      4.2.13                        0    conda-forge
hdf5                      1.10.1                        2    conda-forge
icu                       58.2                 hfc679d8_0    conda-forge
jpeg                      9c                   h470a237_0    conda-forge
jsoncpp                   1.8.1                h2d50403_1    conda-forge
krb5                      1.14.6                        0    conda-forge
libgcc-ng                 7.2.0                hdf63c60_3  
libgfortran               3.0.0                         1  
libiconv                  1.15                 h470a237_1    conda-forge
libnetcdf                 4.6.1                         2    conda-forge
libogg                    1.3.2                         0    conda-forge
libpng                    1.6.34               ha92aebf_1    conda-forge
libssh2                   1.8.0                h5b517e9_2    conda-forge
libstdcxx-ng              7.2.0                hdf63c60_3  
libtheora                 1.1.1                h470a237_1    conda-forge
libtiff                   4.0.9                he6b73bb_1    conda-forge
libuuid                   1.0.3                         1    conda-forge
libvorbis                 1.3.5                         0    conda-forge
libxcb                    1.13                 h470a237_2    conda-forge
libxml2                   2.9.8                h422b904_2    conda-forge
lz4-c                     1.8.1.2                       0    conda-forge
ncurses                   5.9                          10    conda-forge
openssl                   1.0.2o                        0    conda-forge
pip                       18.0                     py36_0    conda-forge
pthread-stubs             0.4                  h470a237_1    conda-forge
python                    3.6.5                         1    conda-forge
readline                  7.0                           0    conda-forge
setuptools                40.0.0                   py36_0    conda-forge
sqlite                    3.20.1                        2    conda-forge
tbb                       2018_20171205                 0    conda-forge
tk                        8.6.8                         0    conda-forge
vtk                       8.1.1           py36hdfe3907_203    conda-forge
wheel                     0.31.1                   py36_0    conda-forge
xorg-kbproto              1.0.7                h470a237_2    conda-forge
xorg-libice               1.0.9                h470a237_4    conda-forge
xorg-libsm                1.2.2                h470a237_4    conda-forge
xorg-libx11               1.6.5                h470a237_2    conda-forge
xorg-libxau               1.0.8                h470a237_6    conda-forge
xorg-libxdmcp             1.1.2                h470a237_7    conda-forge
xorg-libxt                1.1.5                h470a237_1    conda-forge
xorg-xproto               7.0.31               h470a237_7    conda-forge
xz                        5.2.3                         0    conda-forge
zlib                      1.2.11               h470a237_3    conda-forge

If yours doesn't match I would check your .condarc again and make sure nothing is higher than conda-forge. Also, and not to offend, make sure you are actually activating the conda environment. If you do,

import sys
print(sys.path)

Is the conda environment first on that list?

marcelotrevisani commented 6 years ago

Hi, in this env which you are having some problems, could you please run

conda env list

and

conda list --show-channel-urls

and

conda config --show

@akdiem

alexdiem commented 6 years ago

@marcelotrevisani @siboles Sorry, I ended up completely forgetting about this after going on holiday...

It seems like the sys.path is the issue. Even though I definitely activate the environment by running

source activate vtk-error

And seeing (vtk-error) in front of my terminal prompt, print(sys.path) in that environment gives me

['', '/home/alexandra/anaconda3/bin', '/home/alexandra/anaconda3/lib/python36.zip', '/home/alexandra/anaconda3/lib/python3.6', '/home/alexandra/anaconda3/lib/python3.6/lib-dynload', '/home/alexandra/anaconda3/lib/python3.6/site-packages', '/home/alexandra/anaconda3/lib/python3.6/site-packages/IPython/extensions', '/home/alexandra/.ipython']

with no reference to the environment. Why would that happen and how can I fix it?

The output of conda env list is

# conda environments:
#
base                     /home/alexandra/anaconda3
emacs                    /home/alexandra/anaconda3/envs/emacs
fenicsproject            /home/alexandra/anaconda3/envs/fenicsproject
vmtk                     /home/alexandra/anaconda3/envs/vmtk
vtk_error             *  /home/alexandra/anaconda3/envs/vtk_error

The output of conda list --show-channel-urls is

# packages in environment at /home/alexandra/anaconda3/envs/vtk_error:
#
# Name                    Version                   Build  Channel
ca-certificates           2018.8.13            ha4d7672_0    conda-forge
certifi                   2018.8.13                py36_0    conda-forge
curl                      7.60.0                        0    conda-forge
expat                     2.2.5                hfc679d8_1    conda-forge
freetype                  2.8.1                         0    conda-forge
future                    0.16.0                   py36_2    conda-forge
hdf4                      4.2.13                        0    conda-forge
hdf5                      1.10.1                        2    conda-forge
icu                       58.2                 hfc679d8_0    conda-forge
jpeg                      9c                   h470a237_0    conda-forge
jsoncpp                   0.10.6                        1    conda-forge
krb5                      1.14.6                        0    conda-forge
libgcc-ng                 7.2.0                hdf63c60_3    defaults
libgfortran               3.0.0                         1    defaults
libiconv                  1.15                 h470a237_1    conda-forge
libnetcdf                 4.6.1                         2    conda-forge
libogg                    1.3.2                         0    conda-forge
libpng                    1.6.34               ha92aebf_1    conda-forge
libssh2                   1.8.0                h5b517e9_2    conda-forge
libstdcxx-ng              7.2.0                hdf63c60_3    defaults
libtheora                 1.1.1                h470a237_1    conda-forge
libtiff                   4.0.9                he6b73bb_1    conda-forge
libuuid                   1.0.3                         1    conda-forge
libvorbis                 1.3.5                         0    conda-forge
libxcb                    1.13                 h470a237_2    conda-forge
libxml2                   2.9.8                h422b904_2    conda-forge
lz4-c                     1.8.1.2                       0    conda-forge
ncurses                   5.9                          10    conda-forge
openssl                   1.0.2o               h470a237_1    conda-forge
pip                       18.0                     py36_0    conda-forge
pthread-stubs             0.4                  h470a237_1    conda-forge
python                    3.6.5                         1    conda-forge
readline                  7.0                           0    conda-forge
setuptools                40.0.0                   py36_0    conda-forge
sqlite                    3.20.1                        2    conda-forge
tbb                       2018_20171205                 0    conda-forge
tk                        8.6.8                         0    conda-forge
vtk                       8.1.0           py36h3f2a929_203    conda-forge
wheel                     0.31.1                   py36_0    conda-forge
xorg-kbproto              1.0.7                h470a237_2    conda-forge
xorg-libice               1.0.9                h470a237_4    conda-forge
xorg-libsm                1.2.2                h470a237_3    conda-forge
xorg-libx11               1.6.5                h470a237_2    conda-forge
xorg-libxau               1.0.8                h470a237_6    conda-forge
xorg-libxdmcp             1.1.2                h470a237_7    conda-forge
xorg-libxt                1.1.5                h470a237_1    conda-forge
xorg-xproto               7.0.31               h470a237_7    conda-forge
xz                        5.2.3                         0    conda-forge
zlib                      1.2.11               h470a237_3    conda-forge

The output of conda config --show is

add_anaconda_token: True
add_pip_as_python_dependency: True
aggressive_update_packages:
  - ca-certificates
  - certifi
  - openssl
allow_non_channel_urls: False
allow_softlinks: False
always_copy: False
always_softlink: False
always_yes: None
anaconda_upload: None
auto_update_conda: True
changeps1: True
channel_alias: https://conda.anaconda.org
channel_priority: True
channels:
  - conda-forge
  - defaults
client_ssl_cert: None
client_ssl_cert_key: None
clobber: False
create_default_packages: []
custom_channels:
  pkgs/main: https://repo.anaconda.com
  pkgs/free: https://repo.anaconda.com
  pkgs/r: https://repo.anaconda.com
  pkgs/pro: https://repo.anaconda.com
custom_multichannels:
  defaults: ["https://repo.anaconda.com/pkgs/main", "https://repo.anaconda.com/pkgs/free", "https://repo.anaconda.com/pkgs/r", "https://repo.anaconda.com/pkgs/pro"]
  local: []
default_channels:
  - https://repo.anaconda.com/pkgs/main
  - https://repo.anaconda.com/pkgs/free
  - https://repo.anaconda.com/pkgs/r
  - https://repo.anaconda.com/pkgs/pro
disallowed_packages: []
download_only: False
envs_dirs:
  - /home/alexandra/anaconda3/envs
  - /home/alexandra/.conda/envs
force: False
json: False
local_repodata_ttl: 1
max_shlvl: 2
migrated_channel_aliases: []
no_dependencies: False
non_admin_enabled: True
notify_outdated_conda: True
offline: False
override_channels_enabled: True
path_conflict: clobber
pinned_packages: []
pkgs_dirs:
  - /home/alexandra/anaconda3/pkgs
  - /home/alexandra/.conda/pkgs
proxy_servers: {}
quiet: False
remote_connect_timeout_secs: 9.15
remote_max_retries: 3
remote_read_timeout_secs: 60.0
report_errors: None
rollback_enabled: True
safety_checks: warn
shortcuts: True
show_channel_urls: None
ssl_verify: True
track_features: []
use_index_cache: False
use_pip: True
verbosity: 0
whitelist_channels: []
spezold commented 6 years ago

Same problem here, using vtk 8.1.0 from conda-forge channel with Python 3. Fixing jsoncpp to version 1.8.1, also from conda-forge channel, works for me. Therefore, every time I do a conda update --all it has to be followed by conda install jsoncpp=1.8.1 Maybe this helps someone.

Edit: Maybe I should also add that I'm on Ubuntu 16.04

orena1 commented 3 years ago

https://github.com/conda-forge/vtk-feedstock/issues/46#issuecomment-430151894

Same thing here, Windows 10 (tried to install on linux, mac and windows, windows was the first that managed)