Couldn't install pyhsmm on win 10: ERROR: Command errored out with exit status 1

[sinaeslami]

Hi every one. I learned about this amazing hsmm library for python from a scientific paper. I now really need to use it. I tried to install it on my windows 10 laptop and pc. and none of them worked out. I searched every error I got and tried to solve them using previous issues.

GCC

I installed gcc using these recomendations. Then, this is my installed gcc version:

gcc (MinGW.org GCC-6.3.0-1) 6.3.0
Copyright (C) 2016 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Microsoft Visual Studio 14.0

I got errors regarding Ms VS14.0, so I downloaded ms build tools from Microsoft VisualStudio Downloads website and installed the MSVC v14.0 - VS 2015 C++ build tools (v14.00). I tried both enterprise and community version of vs installer.

pyhsmm Installation

then I tried installing the package using pip:

pip install pyhsmm

now here is what I get:

(base) C:\Windows\system32>pip install pyhsmm
Collecting pyhsmm
  Using cached pyhsmm-0.1.7.tar.gz (513 kB)
    ERROR: Command errored out with exit status 1:
     command: 'C:\ProgramData\Anaconda3\python.exe' -c 'import sys, setuptools, tokenize; sys.argv[0] = 
'"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-ox7amkxz\\pyhsmm_322beb62dbe646499d9798cd3d759edd\\setup.py'"'"';
 __file__='"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-
ox7amkxz\\pyhsmm_322beb62dbe646499d9798cd3d759edd\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)
(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base 
'C:\Users\USER\AppData\Local\Temp\pip-pip-egg-info-6ehmj4dx'
         cwd: C:\Users\USER\AppData\Local\Temp\pip-install-ox7amkxz\pyhsmm_322beb62dbe646499d9798cd3d759edd\
    Complete output (26 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\USER\AppData\Local\Temp\pip-install-ox7amkxz\pyhsmm_322beb62dbe646499d9798cd3d759edd\setup.py", line
 76, in <module>
        urlretrieve(eigenurl, eigentarpath)
      File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 247, in urlretrieve
        with contextlib.closing(urlopen(url, data)) as fp:
      File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 222, in urlopen
        return opener.open(url, data, timeout)
      File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 525, in open
        response = self._open(req, data)
      File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 542, in _open
        result = self._call_chain(self.handle_open, protocol, protocol +
      File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 502, in _call_chain
        result = func(*args)
      File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 1383, in http_open
        return self.do_open(http.client.HTTPConnection, req)
      File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 1358, in do_open
        r = h.getresponse()
      File "C:\ProgramData\Anaconda3\lib\http\client.py", line 1347, in getresponse
        response.begin()
      File "C:\ProgramData\Anaconda3\lib\http\client.py", line 307, in begin
        version, status, reason = self._read_status()
      File "C:\ProgramData\Anaconda3\lib\http\client.py", line 276, in _read_status
        raise RemoteDisconnected("Remote end closed connection without"
    http.client.RemoteDisconnected: Remote end closed connection without response
    Downloading Eigen...
    ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/2c/77/6d232566005a1ab9662f903e45122ce4e3679e881a8d0cb0d670c05a49bf/pyhsmm-0.1.7.tar.gz#sha256=3677225656bc39f7d9888361cfeba2c938437ba1df6c8d801985df040fb55f7a 
(from https://pypi.org/simple/pyhsmm/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full 
command output.
  Using cached pyhsmm-0.1.6.tar.gz (491 kB)
    ERROR: Command errored out with exit status 1:
     command: 'C:\ProgramData\Anaconda3\python.exe' -c 'import sys, setuptools, tokenize; sys.argv[0] = 
'"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-ox7amkxz\\pyhsmm_8899810deae1422dbe94c49cf35655af\\setup.py'"'"'; 
__file__='"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-
ox7amkxz\\pyhsmm_8899810deae1422dbe94c49cf35655af\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)
(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base 
'C:\Users\USER\AppData\Local\Temp\pip-pip-egg-info-88pfdwxz'
         cwd: C:\Users\USER\AppData\Local\Temp\pip-install-ox7amkxz\pyhsmm_8899810deae1422dbe94c49cf35655af\
    Complete output (5 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\USER\AppData\Local\Temp\pip-install-ox7amkxz\pyhsmm_8899810deae1422dbe94c49cf35655af\setup.py", line 11, in <module>
        from urllib import urlretrieve
    ImportError: cannot import name 'urlretrieve' from 'urllib' (C:\ProgramData\Anaconda3\lib\urllib\__init__.py)
    ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/9d/91/9a6ea2ac2eb82953b253f3b16967a283ebda324923b4989c34df05de02f8/pyhsmm-0.1.6.tar.gz#sha256=1d000d709f9ba63fbb23e0d509f51dc41b5cc079253d6f4fcb7450a841764c3e (from https://pypi.org/simple/pyhsmm/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
  Using cached pyhsmm-0.1.4.tar.gz (2.0 MB)
Requirement already satisfied: Cython>=0.20.1 in c:\programdata\anaconda3\lib\site-packages (from pyhsmm) (0.29.23)
Requirement already satisfied: numpy in c:\programdata\anaconda3\lib\site-packages (from pyhsmm) (1.20.1)
Requirement already satisfied: scipy in c:\programdata\anaconda3\lib\site-packages (from pyhsmm) (1.6.2)
Requirement already satisfied: matplotlib in c:\programdata\anaconda3\lib\site-packages (from pyhsmm) (3.3.4)
Requirement already satisfied: nose in c:\programdata\anaconda3\lib\site-packages (from pyhsmm) (1.3.7)
Requirement already satisfied: pybasicbayes>=0.1.3 in c:\programdata\anaconda3\lib\site-packages (from pyhsmm) (0.2.2)
Requirement already satisfied: future in c:\programdata\anaconda3\lib\site-packages (from pybasicbayes>=0.1.3->pyhsmm) (0.18.2)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.3 in c:\programdata\anaconda3\lib\site-packages (from matplotlib->pyhsmm) (2.4.7)
Requirement already satisfied: python-dateutil>=2.1 in c:\programdata\anaconda3\lib\site-packages (from matplotlib->pyhsmm) (2.8.1)
Requirement already satisfied: kiwisolver>=1.0.1 in c:\programdata\anaconda3\lib\site-packages (from matplotlib->pyhsmm) (1.3.1)
Requirement already satisfied: pillow>=6.2.0 in c:\programdata\anaconda3\lib\site-packages (from matplotlib->pyhsmm) (8.2.0)
Requirement already satisfied: cycler>=0.10 in c:\programdata\anaconda3\lib\site-packages (from matplotlib->pyhsmm) (0.10.0)
Requirement already satisfied: six in c:\programdata\anaconda3\lib\site-packages (from cycler>=0.10->matplotlib->pyhsmm) (1.15.0)
Building wheels for collected packages: pyhsmm
  Building wheel for pyhsmm (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: 'C:\ProgramData\Anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-ox7amkxz\\pyhsmm_c3eec9d767c041c0aae0bbde31f7b514\\setup.py'"'"'; __file__='"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-ox7amkxz\\pyhsmm_c3eec9d767c041c0aae0bbde31f7b514\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\USER\AppData\Local\Temp\pip-wheel-bkx1ousu'
       cwd: C:\Users\USER\AppData\Local\Temp\pip-install-ox7amkxz\pyhsmm_c3eec9d767c041c0aae0bbde31f7b514\
  Complete output (44 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build\lib.win-amd64-3.8
  creating build\lib.win-amd64-3.8\pyhsmm
  copying pyhsmm\models.py -> build\lib.win-amd64-3.8\pyhsmm
  copying pyhsmm\parallel.py -> build\lib.win-amd64-3.8\pyhsmm
  copying pyhsmm\__init__.py -> build\lib.win-amd64-3.8\pyhsmm
  creating build\lib.win-amd64-3.8\pyhsmm\basic
  copying pyhsmm\basic\abstractions.py -> build\lib.win-amd64-3.8\pyhsmm\basic
  copying pyhsmm\basic\distributions.py -> build\lib.win-amd64-3.8\pyhsmm\basic
  copying pyhsmm\basic\models.py -> build\lib.win-amd64-3.8\pyhsmm\basic
  copying pyhsmm\basic\__init__.py -> build\lib.win-amd64-3.8\pyhsmm\basic
  creating build\lib.win-amd64-3.8\pyhsmm\internals
  copying pyhsmm\internals\hmm_states.py -> build\lib.win-amd64-3.8\pyhsmm\internals
  copying pyhsmm\internals\hsmm_inb_states.py -> build\lib.win-amd64-3.8\pyhsmm\internals
  copying pyhsmm\internals\hsmm_states.py -> build\lib.win-amd64-3.8\pyhsmm\internals
  copying pyhsmm\internals\initial_state.py -> build\lib.win-amd64-3.8\pyhsmm\internals
  copying pyhsmm\internals\transitions.py -> build\lib.win-amd64-3.8\pyhsmm\internals
  copying pyhsmm\internals\__init__.py -> build\lib.win-amd64-3.8\pyhsmm\internals
  creating build\lib.win-amd64-3.8\pyhsmm\plugins
  copying pyhsmm\plugins\__init__.py -> build\lib.win-amd64-3.8\pyhsmm\plugins
  creating build\lib.win-amd64-3.8\pyhsmm\util
  copying pyhsmm\util\general.py -> build\lib.win-amd64-3.8\pyhsmm\util
  copying pyhsmm\util\plot.py -> build\lib.win-amd64-3.8\pyhsmm\util
  copying pyhsmm\util\profiling.py -> build\lib.win-amd64-3.8\pyhsmm\util
  copying pyhsmm\util\stats.py -> build\lib.win-amd64-3.8\pyhsmm\util
  copying pyhsmm\util\testing.py -> build\lib.win-amd64-3.8\pyhsmm\util
  copying pyhsmm\util\text.py -> build\lib.win-amd64-3.8\pyhsmm\util
  copying pyhsmm\util\__init__.py -> build\lib.win-amd64-3.8\pyhsmm\util
  running build_ext
  building 'pyhsmm\internals.hmm_messages_interface' extension
  creating build\temp.win-amd64-3.8
  creating build\temp.win-amd64-3.8\Release
  creating build\temp.win-amd64-3.8\Release\pyhsmm
  creating build\temp.win-amd64-3.8\Release\pyhsmm\internals
  C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Ipyhsmm\deps\Eigen3 -IC:\ProgramData\Anaconda3\lib\site-packages\numpy\core\include -IC:\ProgramData\Anaconda3\include -IC:\ProgramData\Anaconda3\include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.10240.0\ucrt" /EHsc /Tppyhsmm\internals\hmm_messages_interface.cpp /Fobuild\temp.win-amd64-3.8\Release\pyhsmm\internals\hmm_messages_interface.obj -O3 -std=c++11 -DNDEBUG -w -DHMM_TEMPS_ON_HEAP
  cl : Command line warning D9025 : overriding '/W3' with '/w'
  cl : Command line warning D9002 : ignoring unknown option '-O3'
  cl : Command line warning D9002 : ignoring unknown option '-std=c++11'
  hmm_messages_interface.cpp
  C:\ProgramData\Anaconda3\include\pyconfig.h(205): fatal error C1083: Cannot open include file: 'basetsd.h': No such file or directory
  error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\x86_amd64\\cl.exe' failed with exit status 2
  ----------------------------------------
  ERROR: Failed building wheel for pyhsmm
  Running setup.py clean for pyhsmm
Failed to build pyhsmm
Installing collected packages: pyhsmm
    Running setup.py install for pyhsmm ... error
    ERROR: Command errored out with exit status 1:
     command: 'C:\ProgramData\Anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = 
'"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-ox7amkxz\\pyhsmm_c3eec9d767c041c0aae0bbde31f7b514\\setup.py'"'"'; 
__file__='"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-
ox7amkxz\\pyhsmm_c3eec9d767c041c0aae0bbde31f7b514\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)
(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 
'C:\Users\USER\AppData\Local\Temp\pip-record-90tew8g2\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\ProgramData\Anaconda3\Include\pyhsmm'
         cwd: C:\Users\USER\AppData\Local\Temp\pip-install-ox7amkxz\pyhsmm_c3eec9d767c041c0aae0bbde31f7b514\
    Complete output (44 lines):
    running install
    running build
    running build_py
    creating build
    creating build\lib.win-amd64-3.8
    creating build\lib.win-amd64-3.8\pyhsmm
    copying pyhsmm\models.py -> build\lib.win-amd64-3.8\pyhsmm
    copying pyhsmm\parallel.py -> build\lib.win-amd64-3.8\pyhsmm
    copying pyhsmm\__init__.py -> build\lib.win-amd64-3.8\pyhsmm
    creating build\lib.win-amd64-3.8\pyhsmm\basic
    copying pyhsmm\basic\abstractions.py -> build\lib.win-amd64-3.8\pyhsmm\basic
    copying pyhsmm\basic\distributions.py -> build\lib.win-amd64-3.8\pyhsmm\basic
    copying pyhsmm\basic\models.py -> build\lib.win-amd64-3.8\pyhsmm\basic
    copying pyhsmm\basic\__init__.py -> build\lib.win-amd64-3.8\pyhsmm\basic
    creating build\lib.win-amd64-3.8\pyhsmm\internals
    copying pyhsmm\internals\hmm_states.py -> build\lib.win-amd64-3.8\pyhsmm\internals
    copying pyhsmm\internals\hsmm_inb_states.py -> build\lib.win-amd64-3.8\pyhsmm\internals
    copying pyhsmm\internals\hsmm_states.py -> build\lib.win-amd64-3.8\pyhsmm\internals
    copying pyhsmm\internals\initial_state.py -> build\lib.win-amd64-3.8\pyhsmm\internals
    copying pyhsmm\internals\transitions.py -> build\lib.win-amd64-3.8\pyhsmm\internals
    copying pyhsmm\internals\__init__.py -> build\lib.win-amd64-3.8\pyhsmm\internals
    creating build\lib.win-amd64-3.8\pyhsmm\plugins
    copying pyhsmm\plugins\__init__.py -> build\lib.win-amd64-3.8\pyhsmm\plugins
    creating build\lib.win-amd64-3.8\pyhsmm\util
    copying pyhsmm\util\general.py -> build\lib.win-amd64-3.8\pyhsmm\util
    copying pyhsmm\util\plot.py -> build\lib.win-amd64-3.8\pyhsmm\util
    copying pyhsmm\util\profiling.py -> build\lib.win-amd64-3.8\pyhsmm\util
    copying pyhsmm\util\stats.py -> build\lib.win-amd64-3.8\pyhsmm\util
    copying pyhsmm\util\testing.py -> build\lib.win-amd64-3.8\pyhsmm\util
    copying pyhsmm\util\text.py -> build\lib.win-amd64-3.8\pyhsmm\util
    copying pyhsmm\util\__init__.py -> build\lib.win-amd64-3.8\pyhsmm\util
    running build_ext
    building 'pyhsmm\internals.hmm_messages_interface' extension
    creating build\temp.win-amd64-3.8
    creating build\temp.win-amd64-3.8\Release
    creating build\temp.win-amd64-3.8\Release\pyhsmm
    creating build\temp.win-amd64-3.8\Release\pyhsmm\internals
    C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Ipyhsmm\deps\Eigen3 -IC:\ProgramData\Anaconda3\lib\site-packages\numpy\core\include -IC:\ProgramData\Anaconda3\include -IC:\ProgramData\Anaconda3\include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.10240.0\ucrt" /EHsc /Tppyhsmm\internals\hmm_messages_interface.cpp /Fobuild\temp.win-amd64-3.8\Release\pyhsmm\internals\hmm_messages_interface.obj -O3 -std=c++11 -DNDEBUG -w -DHMM_TEMPS_ON_HEAP
    cl : Command line warning D9025 : overriding '/W3' with '/w'
    cl : Command line warning D9002 : ignoring unknown option '-O3'
    cl : Command line warning D9002 : ignoring unknown option '-std=c++11'
    hmm_messages_interface.cpp
    C:\ProgramData\Anaconda3\include\pyconfig.h(205): fatal error C1083: Cannot open include file: 'basetsd.h': No such file or directory
    error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\x86_amd64\\cl.exe' failed with exit status 2
    ----------------------------------------
ERROR: Command errored out with exit status 1: 'C:\ProgramData\Anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-ox7amkxz\\pyhsmm_c3eec9d767c041c0aae0bbde31f7b514\\setup.py'"'"'; __file__='"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-ox7amkxz\\pyhsmm_c3eec9d767c041c0aae0bbde31f7b514\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\USER\AppData\Local\Temp\pip-record-90tew8g2\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\ProgramData\Anaconda3\Include\pyhsmm' Check the logs for full command output.

any help is much appreciated.

[sinaeslami]

Hi every one. I solved the problem and installed the package. here I want to summarize my experience. I have to first, mention that I'm rather new to python and developing and I'm much of a user than a developer; so some of solutions to my problems may seem obvious.

I went through installing this package during past 6 days among my other tasks and gave it a hundred of tries I guess, during which I encountered multiple errors and unexpected not-so-good-documented installation steps.

first of All the command "pip install pyhsmm" didn't work. I even tried "conda install pyhsmm" then it turned out that the library does not exist on any of conda channels.

Then I tried to install it manually using the repository. so I needed "cython" package installed, as well as gcc > 4.3.7 and Ms Build tools.

I pip installed the "Cython", downloaded and installed ms build tools 14.00; but installing the gcc the way that I mentioned in the description comment from this site didn't work. Then I installed gccc #106 issue way.

An important step which I missed but it affected the process, was to make a virtual environment.

so for proper installation, If i want to do it all over again, I'll follow the bellow steps:

1. download and install Microsoft Visual Studio 14.0 like mentioned in the issue description.

2. create a vitual environment. (I'm usein Anaconda jupyter notebook):

   conda create --name your_pyhsmm_env_name python==3.7.9

   higher versions of python is not recommended.

3. install cython:

   conda install cython

4. install gcc:

conda install libpython
conda install -c msys2 m2w64-toolchain

5. install git:

   conda install git

6. clone the repository using git:

git clone https://github.com/mattjj/pyhsmm.git

7. browse to pyhsmm directory and install the package using the setup.py file:

   cd pyhsmm
   python setup.py install

   in this stage you may be done. if there isn't any error regarding gcc, cython, or ms build tools cl.exe file, so go on and test one of examples in the examples folder. Otherwise you may need to go through some of other issues previously raised an answered.

• note 1: in one of my numerous tries I had problem with pyhsmm.internals.hmm_messages_interface. for which the issue #109 may be of help, but for me it didn't and I found out that I hadn't install the cython. so I presume that if you follow the steps I listed, you wouldn't encounter this issue.

• note 2: in another of my tries I went through installing all dependency python packages namely "numpy", "scipy", "matplotlib", "nose", "pybasicbayes >= 0.1.3", "future", "six" manually before running setup.py. One wouldn't need this if above steps followed as well as previous case.