pip install seqvec fails

[tueboesen]

I'm trying to install seqvec, but it fails to install. I'm installing it in a new virtual environment, where the only thing I installed before seqvec was the latest version of pytorch using: pip install torch===1.5.1 torchvision===0.6.1 -f https://download.pytorch.org/whl/torch_stable.html

Afterwards I try to install seqvec, but get the following:

(venv) C:\Users\Tue\PycharmProjects\test>pip install seqvec
Collecting seqvec
  Using cached seqvec-0.3.0-py3-none-any.whl (10 kB)
Requirement already satisfied: numpy<2.0,>=1.17 in c:\users\tue\pycharmprojects\test\venv\lib\site-packages (from seqvec) (1.19.0)
Requirement already satisfied: torch<2.0,>=1.2 in c:\users\tue\pycharmprojects\test\venv\lib\site-packages (from seqvec) (1.5.1)
Collecting allennlp<0.10.0,>=0.9.0
  Using cached allennlp-0.9.0-py3-none-any.whl (7.6 MB)
Collecting tqdm<5.0,>=4.36
  Using cached tqdm-4.47.0-py2.py3-none-any.whl (66 kB)
Collecting gevent==1.4.0
  Using cached gevent-1.4.0.tar.gz (5.2 MB)
Collecting h5py<3.0.0,>=2.10.0
  Using cached h5py-2.10.0-cp38-cp38-win_amd64.whl (2.5 MB)
Requirement already satisfied: future in c:\users\tue\pycharmprojects\test\venv\lib\site-packages (from torch<2.0,>=1.2->seqvec) (0.18.2)
Collecting tensorboardX>=1.2
  Using cached tensorboardX-2.0-py2.py3-none-any.whl (195 kB)
Collecting editdistance
  Downloading editdistance-0.5.3.tar.gz (27 kB)
Collecting boto3
  Using cached boto3-1.14.15-py2.py3-none-any.whl (128 kB)
Collecting overrides
  Using cached overrides-3.1.0.tar.gz (11 kB)
Collecting nltk
  Using cached nltk-3.5.zip (1.4 MB)
Collecting sqlparse>=0.2.4
  Downloading sqlparse-0.3.1-py2.py3-none-any.whl (40 kB)
     |████████████████████████████████| 40 kB 2.5 MB/s
Collecting jsonpickle
  Downloading jsonpickle-1.4.1-py2.py3-none-any.whl (36 kB)
Collecting pytorch-pretrained-bert>=0.6.0
  Using cached pytorch_pretrained_bert-0.6.2-py3-none-any.whl (123 kB)
Collecting scikit-learn
  Using cached scikit_learn-0.23.1-cp38-cp38-win_amd64.whl (6.8 MB)
Collecting spacy<2.2,>=2.1.0
  Using cached spacy-2.1.9.tar.gz (30.7 MB)
  Installing build dependencies ... error
  ERROR: Command errored out with exit status 1:
   command: 'c:\users\tue\pycharmprojects\test\venv\scripts\python.exe' 'c:\users\tue\pycharmprojects\test\venv\lib\site-packages\pip' install --ignore-installed --no-user --prefix 'C:\Users\Tue\AppData\Local\Temp\pip-build-env-6ojquuc8\overlay' --no-warn-script-location --no-binary :none: --only-binary :none: -i
https://pypi.org/simple -- setuptools 'wheel>0.32.0,<0.33.0' Cython 'cymem>=2.0.2,<2.1.0' 'preshed>=2.0.1,<2.1.0' 'murmurhash>=0.28.0,<1.1.0' 'thinc>=7.0.8,<7.1.0'
       cwd: None
  Complete output (66 lines):
  Collecting setuptools
    Using cached setuptools-47.3.1-py3-none-any.whl (582 kB)
  Collecting wheel<0.33.0,>0.32.0
    Using cached wheel-0.32.3-py2.py3-none-any.whl (21 kB)
  Collecting Cython
    Using cached Cython-0.29.20-cp38-cp38-win_amd64.whl (1.7 MB)
  Collecting cymem<2.1.0,>=2.0.2
    Using cached cymem-2.0.3-cp38-cp38-win_amd64.whl (33 kB)
  Collecting preshed<2.1.0,>=2.0.1
    Using cached preshed-2.0.1.tar.gz (113 kB)
  Collecting murmurhash<1.1.0,>=0.28.0
    Using cached murmurhash-1.0.2-cp38-cp38-win_amd64.whl (20 kB)
  Collecting thinc<7.1.0,>=7.0.8
    Using cached thinc-7.0.8.tar.gz (1.9 MB)
  Collecting blis<0.3.0,>=0.2.1
    Using cached blis-0.2.4.tar.gz (1.5 MB)
  Collecting wasabi<1.1.0,>=0.0.9
    Using cached wasabi-0.7.0.tar.gz (22 kB)
  Collecting srsly<1.1.0,>=0.0.6
    Using cached srsly-1.0.2-cp38-cp38-win_amd64.whl (181 kB)
  Collecting numpy>=1.7.0
    Using cached numpy-1.19.0-cp38-cp38-win_amd64.whl (13.0 MB)
  Collecting plac<1.0.0,>=0.9.6
    Using cached plac-0.9.6-py2.py3-none-any.whl (20 kB)
  Collecting tqdm<5.0.0,>=4.10.0
    Using cached tqdm-4.47.0-py2.py3-none-any.whl (66 kB)
  Using legacy setup.py install for preshed, since package 'wheel' is not installed.
  Using legacy setup.py install for thinc, since package 'wheel' is not installed.
  Using legacy setup.py install for blis, since package 'wheel' is not installed.
  Using legacy setup.py install for wasabi, since package 'wheel' is not installed.
  Installing collected packages: setuptools, wheel, Cython, cymem, preshed, murmurhash, numpy, blis, wasabi, srsly, plac, tqdm, thinc
      Running setup.py install for preshed: started
      Running setup.py install for preshed: finished with status 'done'
      Running setup.py install for blis: started
      Running setup.py install for blis: finished with status 'error'
      ERROR: Command errored out with exit status 1:
       command: 'c:\users\tue\pycharmprojects\test\venv\scripts\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\Tue\\AppData\\Local\\Temp\\pip-install-38l5u2ng\\blis\\setup.py'"'"'; __file__='"'"'C:\\Users\\Tue\\AppData\\Local\\Temp\\pip-install-38l5u2ng\\blis\\setup.py'"'"';f=ge
tattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\Tue\AppData\Local\Temp\pip-record-5m63lxdn\install-record.txt' --single-version-externally-managed --prefix 'C:\Users\Tue\AppData\L
ocal\Temp\pip-build-env-6ojquuc8\overlay' --compile --install-headers 'c:\users\tue\pycharmprojects\test\venv\include\site\python3.8\blis'
           cwd: C:\Users\Tue\AppData\Local\Temp\pip-install-38l5u2ng\blis\
      Complete output (25 lines):
      BLIS_COMPILER? None
      running install
      running build
      running build_py
      creating build
      creating build\lib.win-amd64-3.8
      creating build\lib.win-amd64-3.8\blis
      copying blis\about.py -> build\lib.win-amd64-3.8\blis
      copying blis\benchmark.py -> build\lib.win-amd64-3.8\blis
      copying blis\__init__.py -> build\lib.win-amd64-3.8\blis
      creating build\lib.win-amd64-3.8\blis\tests
      copying blis\tests\common.py -> build\lib.win-amd64-3.8\blis\tests
      copying blis\tests\test_dotv.py -> build\lib.win-amd64-3.8\blis\tests
      copying blis\tests\test_gemm.py -> build\lib.win-amd64-3.8\blis\tests
      copying blis\tests\__init__.py -> build\lib.win-amd64-3.8\blis\tests
      copying blis\cy.pyx -> build\lib.win-amd64-3.8\blis
      copying blis\py.pyx -> build\lib.win-amd64-3.8\blis
      copying blis\cy.pxd -> build\lib.win-amd64-3.8\blis
      copying blis\__init__.pxd -> build\lib.win-amd64-3.8\blis
      running build_ext
      error: [WinError 2] The system cannot find the file specified
      msvc
      py_compiler msvc
      {'LS_COLORS': 'rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:
*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31
:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;
35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:
*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:', 'HOSTTYPE': 'x86_64', 'LESSCLOSE': '/usr/bin/l
esspipe %s %s', 'LANG': 'C.UTF-8', 'OLDPWD': '/home/matt/repos/flame-blis', 'VIRTUAL_ENV': '/home/matt/repos/cython-blis/env3.6', 'USER': 'matt', 'PWD': '/home/matt/repos/cython-blis', 'HOME': '/home/matt', 'NAME': 'LAPTOP-OMKOB3VM', 'XDG_DATA_DIRS': '/usr/local/share:/usr/share:/var/lib/snapd/desktop', 'SHELL': '
/bin/bash', 'TERM': 'xterm-256color', 'SHLVL': '1', 'LOGNAME': 'matt', 'PATH': '/home/matt/repos/cython-blis/env3.6/bin:/tmp/google-cloud-sdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/mnt/c/Users/matt/Documents/cmder/vendor/conemu-maximus5/ConEmu/Scripts:/mnt/c/U
sers/matt/Documents/cmder/vendor/conemu-maximus5:/mnt/c/Users/matt/Documents/cmder/vendor/conemu-maximus5/ConEmu:/mnt/c/Python37/Scripts:/mnt/c/Python37:/mnt/c/Program Files (x86)/Intel/Intel(R) Management Engine Components/iCLS:/mnt/c/Program Files/Intel/Intel(R) Management Engine Components/iCLS:/mnt/c/Windows/S
ystem32:/mnt/c/Windows:/mnt/c/Windows/System32/wbem:/mnt/c/Windows/System32/WindowsPowerShell/v1.0:/mnt/c/Program Files (x86)/Intel/Intel(R) Management Engine Components/DAL:/mnt/c/Program Files/Intel/Intel(R) Management Engine Components/DAL:/mnt/c/Program Files (x86)/Intel/Intel(R) Management Engine Components/I
PT:/mnt/c/Program Files/Intel/Intel(R) Management Engine Components/IPT:/mnt/c/Program Files/Intel/WiFi/bin:/mnt/c/Program Files/Common Files/Intel/WirelessCommon:/mnt/c/Program Files (x86)/NVIDIA Corporation/PhysX/Common:/mnt/c/ProgramData/chocolatey/bin:/mnt/c/Program Files/Git/cmd:/mnt/c/Program Files/LLVM/bin:
/mnt/c/Windows/System32:/mnt/c/Windows:/mnt/c/Windows/System32/wbem:/mnt/c/Windows/System32/WindowsPowerShell/v1.0:/mnt/c/Windows/System32/OpenSSH:/mnt/c/Program Files/nodejs:/mnt/c/Users/matt/AppData/Local/Microsoft/WindowsApps:/mnt/c/Users/matt/AppData/Local/Programs/Microsoft VS Code/bin:/mnt/c/Users/matt/AppDa
ta/Roaming/npm:/snap/bin:/mnt/c/Program Files/Oracle/VirtualBox', 'PS1': '(env3.6) \\[\\e]0;\\u@\\h: \\w\\a\\]${debian_chroot:+($debian_chroot)}\\[\\033[01;32m\\]\\u@\\h\\[\\033[00m\\]:\\[\\033[01;34m\\]\\w\\[\\033[00m\\]\\$ ', 'VAGRANT_HOME': '/home/matt/.vagrant.d/', 'LESSOPEN': '| /usr/bin/lesspipe %s', '_': '/
home/matt/repos/cython-blis/env3.6/bin/python'}
      clang -c C:\Users\Tue\AppData\Local\Temp\pip-install-38l5u2ng\blis\blis\_src\config\bulldozer\bli_cntx_init_bulldozer.c -o C:\Users\Tue\AppData\Local\Temp\tmpggx9vkn0\bli_cntx_init_bulldozer.o -O2 -funroll-all-loops -std=c99 -D_POSIX_C_SOURCE=200112L -DBLIS_VERSION_STRING="0.5.0-6" -DBLIS_IS_BUILDING_LIBRARY
 -Iinclude\windows-x86_64 -I.\frame\3\ -I.\frame\ind\ukernels\ -I.\frame\1m\ -I.\frame\1f\ -I.\frame\1\ -I.\frame\include -IC:\Users\Tue\AppData\Local\Temp\pip-install-38l5u2ng\blis\blis\_src\include\windows-x86_64
      ----------------------------------------
  ERROR: Command errored out with exit status 1: 'c:\users\tue\pycharmprojects\test\venv\scripts\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\Tue\\AppData\\Local\\Temp\\pip-install-38l5u2ng\\blis\\setup.py'"'"'; __file__='"'"'C:\\Users\\Tue\\AppData\\Local\\Temp\\pip-install-3
8l5u2ng\\blis\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\Tue\AppData\Local\Temp\pip-record-5m63lxdn\install-record.txt' --single-version-externally-managed
 --prefix 'C:\Users\Tue\AppData\Local\Temp\pip-build-env-6ojquuc8\overlay' --compile --install-headers 'c:\users\tue\pycharmprojects\test\venv\include\site\python3.8\blis' Check the logs for full command output.
  ----------------------------------------
ERROR: Command errored out with exit status 1: 'c:\users\tue\pycharmprojects\test\venv\scripts\python.exe' 'c:\users\tue\pycharmprojects\test\venv\lib\site-packages\pip' install --ignore-installed --no-user --prefix 'C:\Users\Tue\AppData\Local\Temp\pip-build-env-6ojquuc8\overlay' --no-warn-script-location --no-bin
ary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools 'wheel>0.32.0,<0.33.0' Cython 'cymem>=2.0.2,<2.1.0' 'preshed>=2.0.1,<2.1.0' 'murmurhash>=0.28.0,<1.1.0' 'thinc>=7.0.8,<7.1.0' Check the logs for full command output.

It seems that spacy was the first package that failed to install, so I tried installing that one manually, which went fine, but then when I tried to install seqvec again, it still fails. My guess is that some of the package requirements are incompatible (Likely some of the packages have too old requirements and should be updated to newer versions)

Any advice for how to fix this, so I can try to use seqvec for my research would be greatly appreciated.

[sacdallago]

Right away option: use the docker container ~https://hub.docker.com/repository/docker/rostlab/seqvec~

EDIT: I put in the wrong link, here it is: https://hub.docker.com/r/rostlab/seqvec

[tueboesen]

I have never used docker before, but I will try and check it out now, thank you for the quick reply.

[konstin]

Could you just try pip install seqvec in a new virtualenv? Also could you please post the output of pip debug?

[tueboesen]

Could you just try pip install seqvec in a new virtualenv? Also could you please post the output of pip debug?

That was what I just did. I initially tried to install it in my standard machine learning environment so I could integrate it into my previous work, but when that failed I started a new environment. Where I tried to install seqvec:

(venv) C:\Users\Tue\PycharmProjects\test>pip install seqvec
Collecting seqvec
  Using cached seqvec-0.3.0-py3-none-any.whl (10 kB)
ERROR: Could not find a version that satisfies the requirement torch<2.0,>=1.2 (from seqvec) (from versions: 0.1.2, 0.1.2.post1, 0.1.2.post2)
ERROR: No matching distribution found for torch<2.0,>=1.2 (from seqvec)

So I manually installed torch as explained above, and then I tried again, which gave the result I posted in the original comment.

I will try the install again with debug on, so I can get more details.

[tueboesen]

Here is the result of pip debug:

Is it because I'm using Python 3.8 that I'm getting these problems, I see that you guys mention 3.7 in your FAQ, I just figured that was the minimum required version, but it might be the maximum required version?

(venv) C:\Users\Tue\PycharmProjects\test>pip debug --verbose
WARNING: This command is only meant for debugging. Do not use this with automation for parsing and getting these details, since the output and options of this command may change without notice.
pip version: pip 20.1.1 from c:\users\tue\pycharmprojects\test\venv\lib\site-packages\pip (python 3.8)
sys.version: 3.8.2 (tags/v3.8.2:7b3ab59, Feb 25 2020, 23:03:10) [MSC v.1916 64 bit (AMD64)]
sys.executable: c:\users\tue\pycharmprojects\test\venv\scripts\python.exe
sys.getdefaultencoding: utf-8
sys.getfilesystemencoding: utf-8
locale.getpreferredencoding: cp1252
sys.platform: win32
sys.implementation:
  name: cpython
'cert' config value: Not specified
REQUESTS_CA_BUNDLE: None
CURL_CA_BUNDLE: None
pip._vendor.certifi.where(): c:\users\tue\pycharmprojects\test\venv\lib\site-packages\pip\_vendor\certifi\cacert.pem
pip._vendor.DEBUNDLED: False
vendored library versions:
  appdirs==1.4.3
  CacheControl==0.12.6
  colorama==0.4.3
  contextlib2==0.6.0.post1 (Unable to locate actual module version, using vendor.txt specified version)
  distlib==0.3.0
  distro==1.5.0 (Unable to locate actual module version, using vendor.txt specified version)
  html5lib==1.0.1
  ipaddress==1.0.23
  msgpack==1.0.0 (Unable to locate actual module version, using vendor.txt specified version)
  packaging==20.3
  pep517==0.8.2
  progress==1.5
  pyparsing==2.4.7
  requests==2.23.0
  certifi==2020.04.05.1
  chardet==3.0.4
  idna==2.9
  urllib3==1.25.8
  resolvelib==0.3.0
  retrying==1.3.3 (Unable to locate actual module version, using vendor.txt specified version)
  setuptools==44.0.0 (Unable to locate actual module version, using vendor.txt specified version)
  six==1.14.0
  toml==0.10.0
  webencodings==0.5.1 (Unable to locate actual module version, using vendor.txt specified version)
Compatible tags: 30
  cp38-cp38-win_amd64
  cp38-abi3-win_amd64
  cp38-none-win_amd64
  cp37-abi3-win_amd64
  cp36-abi3-win_amd64
  cp35-abi3-win_amd64
  cp34-abi3-win_amd64
  cp33-abi3-win_amd64
  cp32-abi3-win_amd64
  py38-none-win_amd64
  py3-none-win_amd64
  py37-none-win_amd64
  py36-none-win_amd64
  py35-none-win_amd64
  py34-none-win_amd64
  py33-none-win_amd64
  py32-none-win_amd64
  py31-none-win_amd64
  py30-none-win_amd64
  cp38-none-any
  py38-none-any
  py3-none-any
  py37-none-any
  py36-none-any
  py35-none-any
  py34-none-any
  py33-none-any
  py32-none-any
  py31-none-any
  py30-none-any

[tueboesen]

I did some further testing and made a new virtual environment with Python 3.7 where I installed torch version 1.2 and with this old version of torch the installation worked.

Now the big question is whether the project I want to integrate this into can run on that old version of torch. Do you have any plans of updating the project so it can run on newer versions of torch/python? or what would you guys suggest for that?

[mheinzinger]

Great to hear that you managed to run SeqVec. Regarding your project planning: we will release a new set of pre-trained LMs for proteins very soon (end of this week, latest next week). I would recommend to simply switch to those newer LMs once we've put them online.