greenelab / tybalt

Training and evaluating a variational autoencoder for pan-cancer gene expression data
BSD 3-Clause "New" or "Revised" License
162 stars 62 forks source link

Error when setting up environment #154

Closed matcyr closed 3 years ago

matcyr commented 3 years ago

Hi Dr. Greg,

I tried to implement your code but was trapped when creating environment using environment.yml you provided. There are some conflicts for the dependency. I am not sure whether it is rasied by the reason the version might be too old. Will you update the dependencies?

Thanks yurui

gwaybio commented 3 years ago

Thanks @matcyr, and congrats on filing your first issue - welcome!

Can you paste your error message after running conda env create --force --file environment.yml?

matcyr commented 3 years ago

Thanks @matcyr, and congrats on filing your first issue - welcome!

Can you paste your error message after running conda env create --force --file environment.yml?

The error code is here: (base) :~/tybalt$ conda env create --force --file environment.yml Warning: you have pip-installed dependencies in your environment file, but you do not list pip itself as one of your conda dependencies. Conda may not use the correct pip to install your packages, and they may end up in the wrong place. Please add an explicit pip dependency. I'm adding one for you, but still nagging you. Collecting package metadata (repodata.json): done Solving environment: - Found conflicts! Looking for incompatible packages. This can take several minutes. Press CTRL-C to abort. Examining conflict for nb_conda_kernels widgetsnbextension ipywidgets ipykernel jupyter: : 28it [01:3Examining conflict for ipywidgets widgetsnbextension ipykernel: : 29it [01:31, 2.03s/it] Examining conflict for widgetsnbextension tensorflow ipywidgets ipykernel keras: : 33it [01:33, 1.19Examining conflict for widgetsnbextension tensorflow ipywidgets ipykernel keras: : 34it [01:33, 1.12Examining conflict for jupyter ipywidgets widgetsnbextension nb_conda_kernels: : 34it [01:34, 1.12itExamining conflict for jupyter ipywidgets widgetsnbextension nb_conda_kernels: : 35it [01:34, 1.06itExamining conflict for jupyter ipywidgets widgetsnbextension: : 35it [01:35, 1.06it/s] Examining conflict for jupyter ipykernel widgetsnbextension nb_conda_kernels: : 44it [01:39, 1.39it/Examining conflict for jupyter widgetsnbextension nb_conda_kernels: : 45it [01:40, 1.39it/s] failed

For instance, I change conda-forge::keras=2.1.3 to defaults::keras=2.1.3 since keras 2.1.3 is not in channel conda forge right now. I think the conflit happens when installing matplotlib and keras-tqdm . Then, I tried to install line by line. I tried to use: 'conda install conda-forge::tensorflow=1.4.0' then import tensorflow and get Traceback (most recent call last): File "", line 1, in File "/home/guoyue/anaconda3/envs/tmp/lib/python3.5/site-packages/tensorflow/init.py", line 24, in from tensorflow.python import File "/home/guoyue/anaconda3/envs/tmp/lib/python3.5/site-packages/tensorflow/python/init.py", line 52, in from tensorflow.core.framework.graph_pb2 import File "/home/guoyue/anaconda3/envs/tmp/lib/python3.5/site-packages/tensorflow/core/framework/graph_pb2.py", line 6, in from google.protobuf import descriptor as _descriptor File "/home/guoyue/anaconda3/envs/tmp/lib/python3.5/site-packages/google/protobuf/descriptor.py", line 47, in from google.protobuf.pyext import _message ImportError: /home/guoyue/anaconda3/envs/tmp/lib/python3.5/site-packages/google/protobuf/pyext/_message.cpython-35m-x86_64-linux-gnu.so: undefined symbol: _ZNK6google8protobuf10TextFormat17FieldValuePrinter9PrintBoolEb I could not figure out why it happens. It never happened to me when I used tensoflow 2.0 and above.

Thanks, yurui

gwaybio commented 3 years ago

Thanks @matcyr - keras and tensorflow have been notoriously difficult to support using conda, although it is surprising that conda-forge no longer supports that keras version. But since you found it in defaults, maybe it isn't a big deal?

I think the conflit happens when installing matplotlib and keras-tqdm .

Try removing keras-tqdm and then recreating the environment. I am ok with removing this package in favor of more stable maintenance.

matcyr commented 3 years ago

Thanks @matcyr - keras and tensorflow have been notoriously difficult to support using conda, although it is surprising that conda-forge no longer supports that keras version. But since you found it in defaults, maybe it isn't a big deal?

I think the conflit happens when installing matplotlib and keras-tqdm .

Try removing keras-tqdm and then recreating the environment. I am ok with removing this package in favor of more stable maintenance.

I removed keras-tqdm and successfuly create the environment. However I still get trouble when importing packages. I got the error report when import tensorflow: ImportError Traceback (most recent call last)

in ----> 1 import tensorflow ~/anaconda3/envs/tybalt/lib/python3.5/site-packages/tensorflow/__init__.py in 22 23 # pylint: disable=wildcard-import ---> 24 from tensorflow.python import * 25 # pylint: enable=wildcard-import 26 ~/anaconda3/envs/tybalt/lib/python3.5/site-packages/tensorflow/python/__init__.py in 50 51 # Protocol buffers ---> 52 from tensorflow.core.framework.graph_pb2 import * 53 from tensorflow.core.framework.node_def_pb2 import * 54 from tensorflow.core.framework.summary_pb2 import * ~/anaconda3/envs/tybalt/lib/python3.5/site-packages/tensorflow/core/framework/graph_pb2.py in 4 import sys 5 _b=sys.version_info[0]<3 and (lambda x:x) or (lambda x:x.encode('latin1')) ----> 6 from google.protobuf import descriptor as _descriptor 7 from google.protobuf import message as _message 8 from google.protobuf import reflection as _reflection ~/anaconda3/envs/tybalt/lib/python3.5/site-packages/google/protobuf/descriptor.py in 45 import binascii 46 import os ---> 47 from google.protobuf.pyext import _message 48 _USE_C_DESCRIPTORS = getattr(_message, '_USE_C_DESCRIPTORS', False) 49 ImportError: /home/guoyue/anaconda3/envs/tybalt/lib/python3.5/site-packages/google/protobuf/pyext/_message.cpython-35m-x86_64-linux-gnu.so: undefined symbol: _ZNK6google8protobuf10TextFormat17FieldValuePrinter9PrintBoolEb May I know which OS you are using? I am working on ubuntu 18.04 and conda version 4.9.2. Will you find this error if you recreating the environment right now? Thanks for the time.
matcyr commented 3 years ago

I am able to set up the environment by changing pip install to python -m pip install. I will close the issue. Thank you.

yagmuronay commented 3 years ago

Hi @matcyr,

I have the same problem. I set up the environment step by step however I get the import error below. Could you provide me the exact steps and environment file that worked for you? Thank you in advance.

    import tensorflow as tf
  File "/mnt/lsf-nas-1/os-shared/anaconda3/envs/tybalt/lib/python3.5/site-packages/tensorflow/__init__.py", line 24, in <module>
    from tensorflow.python import *
  File "/mnt/lsf-nas-1/os-shared/anaconda3/envs/tybalt/lib/python3.5/site-packages/tensorflow/python/__init__.py", line 54, in <module>
    from tensorflow.core.framework.graph_pb2 import *
  File "/mnt/lsf-nas-1/os-shared/anaconda3/envs/tybalt/lib/python3.5/site-packages/tensorflow/core/framework/graph_pb2.py", line 6, in <module>
    from google.protobuf import descriptor as _descriptor
  File "/mnt/lsf-nas-1/os-shared/anaconda3/envs/tybalt/lib/python3.5/site-packages/google/protobuf/descriptor.py", line 47, in <module>
    from google.protobuf.pyext import _message
ImportError: /mnt/lsf-nas-1/os-shared/anaconda3/envs/tybalt/lib/python3.5/site-packages/google/protobuf/pyext/_message.cpython-35m-x86_64-linux-gnu.so: undefined symbol: _ZNK6google8protobuf10TextFormat17FieldValuePrinter9PrintBoolEb
yagmuronay commented 3 years ago

pip install --upgrade --force-reinstall protobuf solved it for me.