Installation Error - Githubissues

iPFAS commented 1 year ago

Describe the bug I tried to install the GPU version, but there were many software package dependency conflicts. Even after I manually installed it, the problem still could not be solved.For example, the following pip command. There are many such problems that prevent me from using such an excellent framework for scientific research, which is regrettable.

To Reproduce

pip install git+https://github.com/PaccMann/paccmann_generator@0.0.2 --user

jannisborn commented 1 year ago

Hi @iPFAS, Thanks for getting in touch. Can you please report the commands that you executed? We have a thorough test suite for Ubuntu and most of the developers use Mac, so I'm a bit surprised to see this

Note that generally, you dont have to install any packages manually, just follow the installation workflow as described in the README

jannisborn commented 1 year ago

We also need to know which gt4sd version you're trying to install and which OS you use.

iPFAS commented 1 year ago

Thank you very much for your prompt reply. I'm sorry, it may be partly due to my system environment problem. The description above is not clear enough and has caused you trouble. Here are my specific installation steps."。😊

I use the system: centos stream.
The latest version is installed, which is the current gt4sd-core repository

Then I said to follow the README.md for installation, the command is as follows:

git clone https://github.com/GT4SD/gt4sd-core.git 
cd gt4sd-core/ 
conda env create -f conda_gpu.yml 
conda activate gt4sd

During the process, I encountered the following error.

    AttributeError: module 'torch' has no attribute 'autocast'

This may be a problem with the torch version, but GT4SD requires pytorch>=1.0,<=1.12.1=cu. I feel that if I upgrade the pytorch version, there will also be conflicts.

conda list

  _libgcc_mutex             0.1                 conda_forge    conda-forge
  _openmp_mutex             4.5                  2_kmp_llvm    conda-forge
  blas                      1.0                         mkl    http://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
  brotlipy                  0.7.0           py38h0a891b7_1005    conda-forge
  bzip2                     1.0.8                h7f98852_4    conda-forge
  ca-certificates           2022.12.7            ha878542_0    conda-forge
  certifi                   2022.12.7          pyhd8ed1ab_0    conda-forge
  cffi                      1.15.1           py38h4a40e3a_3    conda-forge
  charset-normalizer        3.1.0              pyhd8ed1ab_0    conda-forge
  cryptography              40.0.2           py38h3d167d9_0    conda-forge
  cudatoolkit               11.6.0               habf752d_9    nvidia
  ffmpeg                    4.3                  hf484d3e_0    pytorch
  freetype                  2.12.1               hca18f0e_1    conda-forge
  gmp                       6.2.1                h58526e2_0    conda-forge
  gnutls                    3.6.13               h85f3911_1    conda-forge
  icu                       72.1                 hcb278e6_0    conda-forge
  idna                      3.4                pyhd8ed1ab_0    conda-forge
  jpeg                      9e                   h0b41bf4_3    conda-forge
  lame                      3.100             h166bdaf_1003    conda-forge
  lcms2                     2.15                 hfd0df8a_0    conda-forge
  ld_impl_linux-64          2.40                 h41732ed_0    conda-forge
  lerc                      4.0.0                h27087fc_0    conda-forge
  libdeflate                1.17                 h0b41bf4_0    conda-forge
  libffi                    3.4.2                h7f98852_5    conda-forge
  libgcc-ng                 12.2.0              h65d4601_19    conda-forge
  libhwloc                  2.9.1                hd6dc26d_0    conda-forge
  libiconv                  1.17                 h166bdaf_0    conda-forge
  libnsl                    2.0.0                h7f98852_0    conda-forge
  libpng                    1.6.39               h753d276_0    conda-forge
  libsqlite                 3.40.0               h753d276_1    conda-forge
  libstdcxx-ng              12.2.0              h46fd767_19    conda-forge
  libtiff                   4.5.0                h6adf6a1_2    conda-forge
  libuuid                   2.38.1               h0b41bf4_0    conda-forge
  libwebp-base              1.3.0                h0b41bf4_0    conda-forge
  libxcb                    1.13              h7f98852_1004    conda-forge
  libxml2                   2.10.4               hfdac1af_0    conda-forge
  libzlib                   1.2.13               h166bdaf_4    conda-forge
  llvm-openmp               16.0.2               h4dfa4b3_0    conda-forge
  mkl                       2021.4.0           h8d4b97c_729    conda-forge
  mkl-service               2.4.0            py38h95df7f1_0    conda-forge
  mkl_fft                   1.3.1            py38h8666266_1    conda-forge
  mkl_random                1.2.2            py38h1abd341_0    conda-forge
  ncurses                   6.3                  h27087fc_1    conda-forge
  nettle                    3.6                  he412f7d_0    conda-forge
  numpy                     1.23.5           py38h14f4228_0    http://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
  numpy-base                1.23.5           py38h31eccc5_0    http://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
  openh264                  2.1.1                h780b84a_0    conda-forge
  openjpeg                  2.5.0                hfec8fc6_2    conda-forge
  openssl                   3.1.0                hd590300_2    conda-forge
  pillow                    9.4.0            py38hde6dc18_1    conda-forge
  pip                       20.2.4                     py_0    conda-forge
  pthread-stubs             0.4               h36c2ea0_1001    conda-forge
  pycparser                 2.21               pyhd8ed1ab_0    conda-forge
  pyopenssl                 23.1.1             pyhd8ed1ab_0    conda-forge
  pysocks                   1.7.1              pyha2e5f31_6    conda-forge
  python                    3.8.16          he550d4f_1_cpython    conda-forge
  python_abi                3.8                      3_cp38    conda-forge
  pytorch                   1.12.1          py3.8_cuda11.6_cudnn8.3.2_0    pytorch
  pytorch-mutex             1.0                        cuda    pytorch
  pytorch-scatter           2.0.9           py38_torch_1.12.0_cu116    pyg
  readline                  8.2                  h8228510_1    conda-forge
  requests                  2.28.2             pyhd8ed1ab_1    conda-forge
  setuptools                67.7.2             pyhd8ed1ab_0    conda-forge
  six                       1.16.0             pyh6c4a22f_0    conda-forge
  tbb                       2021.9.0             hf52228f_0    conda-forge
  tk                        8.6.12               h27826a3_0    conda-forge
  torchaudio                0.12.1               py38_cu116    pytorch
  torchvision               0.13.1               py38_cu116    pytorch
  typing_extensions         4.5.0              pyha770c72_0    conda-forge
  urllib3                   1.26.15            pyhd8ed1ab_0    conda-forge
  wheel                     0.40.0             pyhd8ed1ab_0    conda-forge
  xorg-libxau               1.0.9                h7f98852_0    conda-forge
  xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
  xz                        5.2.6                h166bdaf_0    conda-forge
  zlib                      1.2.13               h166bdaf_4    conda-forge

GPU： Of course, considering that this may be a problem with my environment, I used manual installation. That is, according to the content in conda_gpu.yml, install the packages separately. However, when I execute pip install -r vcs_requirements.txt, there will be package dependency conflicts as shown in the screenshot below.

So I checked one by one why there was a conflict and found that as pip prompted, for paccmann_generator, the version of pytoda it needs is 0.1.1, while toxsmi requires pytoda version >= 1.1.2." Specifically as shown in the screenshot. paccmann_generator requirements.txt https://github.com/PaccMann/paccmann_generator/blob/master/requirements.txt QQ截图20230425200846 toxsmi requirements.txt https://github.com/PaccMann/toxsmi QQ截图20230425200520 Considering your professionalism, I feel that it is most likely an issue with my environment. I apologize for not providing a detailed description in my question. Thank you for your support.

jannisborn commented 1 year ago

Ok, thx for sharing those details. Let's make a step back: after activating the env, it seems that you did not pip install gt4sd or pip install -e .? This would clearly be needed in the first place before you can use any GT4SD (including the trainer)

iPFAS commented 1 year ago

Sorry for the late reply. The main reason is that the installation process for reproducing the issue takes too long. There is a detailed explanation below. Of course, I didn’t miss this step. In fact I’ve tried to install it many times before, but I couldn’t get it done and that’s why I submitted an issue.

I replaced it with another CentOS Stream server that has a cleaner installation environment to reproduce my issue. The process is as follows.

    sudo conda env create -f conda_gpu.yml
    sudo conda activate gt4sd
    pip --no-cache-dir install gt4sd
    pip install -r vcs_requirements.txt

sudo conda env create -f conda_gpu.yml ---- executed successfully without any issues.
pip --no-cache-dir install gt4sd ---- raise an error ：ImportError: PyTorch is required to install pytorch-fast-transformers. Please install your favorite version of PyTorch, we support 1.3.1, 1.5.0 and >=1.6 The installation log shows that PyTorch was installed. I’m not sure why there is an error here. Perhaps only torch was installed but not PyTorch, which caused this? _Collecting torch Downloading torch-1.13.1-cp37-cp37m-manylinux1_x8664.whl (887.5 MB)
This installation step requires a lot of time because pip needs to search many versions of the package.
another error
pip install -r vcs_requirements.txt ---- There is still an error caused by version dependencies here Of course, these issues are relatively easy to solve, so I upgraded them one by one according to the prompts
```
pip install scikit-learn==0.23.2
pip install importlib-resources==5.10.0
pip install numpy==1.21
```
then, new errors about version dependencies occured and the new requirement error of numpy : yellowbrick 1.3.post1 requires numpy<1.20,>=1.16.0, but you'll have numpy 1.21.0 which is incompatible. that means ：after I upgraded numpy, another package prompted me that the version I upgraded exceeded the requirements, which cannot be solved. Of course, sometimes the version issue of numpy can be ignored and does not affect the use of the program. I ignored it and continued.
So I continued to upgrade the versions according to the prompts
```
pip install protobuf==3.19.6
pip install scikit-learn==1.0.0
```
There are still two errors left. I decided to ignore them and test using gt4sd-inference --help or gt4sd-trainer --help
```
gt4sd-inference --help
```
However, after experiencing so many difficulties and obstacles, I still cannot run it successfully. The prompt of Segmentation fault (core dumped) makes me heartbroken. Looking at the error message, it is probably a problem with tensorflow. At this point, I may have to solve the tensorflow environment problem again.

jannisborn commented 1 year ago

Hi @iPFAS,

Sorry that you are experience such issues! Let's go step by step.

sudo conda env create -f conda_gpu.yml --> works
pip --no-cache-dir install gt4sd --> This is a red flag. VERY surprising that pytorch seems to not be installed, especially since the previous command was successful. If you check the conda_gpu.yml file you'll see that it should take care of the pytorch installation. Did you manually verify that pytorch was indeed not installed after step 1?
pip install -r vcs_requirements.txt --> This should not be necessary since it's being executed from inside the conda_gpu.yml. Generally these version mismatch log messages are more warnings than actual errors, this is just pip telling you about potential problem sources. However, when I set up the env with the current tip of master, I do not see such warnings.
gt4sd-inference --help --> Thx for reporting the segfault! It's interesting, I've seen this issue in the past it's caused by pytorch-lightning. It's a common issue described here: https://github.com/Lightning-AI/lightning/issues/11663 We thought that we had mitigated that by fixing the relation with sentencepice. Can you check whether you get a segfault if you open python and just import pytorch lightning? Probably yes. Probably, those parts of the library that do not import lightning are functional. That's obviously not a permanent solution. We're looking into this currently and try to fix it.

So overall, from all the things you report only 4) is something that we can reproduce. Which version of conda do you have? We use 4.12.0, you might experience issues with much more recent versions

iPFAS commented 1 year ago

Thank you for your help. I followed your advice and downgraded conda to 4.12.0 and tried again. Unfortunately, I still cannot install it successfully. Overall, it is probably due to my network and environment. Considering the requirements of work progress, I can only try again later when I have time. Of course, thank you very much for your timely and professional reply.I will close the issue Thank you again.

jannisborn commented 1 year ago

Hi @iPFAS,

We just released a new version (1.3.1) which fixes some issues that multiple users observed in the installation/setup including the segmentation fault you reported. Please feel free to retry the installation with the latest version! Thanks for your patience

iPFAS commented 1 year ago

Hi， Thank you for letting me know about the new version and the fixes it includes. I appreciate your team’s efforts in addressing the issues and will definitely consider retrying the installation with the latest version. Thanks for your support

GT4SD / gt4sd-core

Installation Error #216