Failed install on Windows 10 with Python 3.5.2

[woakesd]

I'm not able to install using the command line command

py -m pip install ahocorasick

The compile is failing, I've attached the error messages I'm getting.

I have Visual Studio Professional 2015 Professional installed and Cython 0.25.2. Any pointers on what I may be doing wrong gratefully received!

ahocorasick-compile-error.txt

[pombredanne]

@woakesd ... hum on Python 3.5? FYI Cython is not used. @WojciechMula The current master on Appveyor here has the same failure: https://ci.appveyor.com/project/WojciechMula/pyahocorasick/build/job/76ke93wsrmua38l2

[woakesd]

Had a look at the code, in windows.h some of the types defined in stdint.h (a Visual Studio include file since VCS2010 I think) but not the type uint32_t which is one of the failures.

Perhaps windows.h should include stdint.h?

[woakesd]

I make the following changes to windows.h and it compiles and appears to work, but I couldn't run the tests in windows.

diff --git a/windows.h b/windows.h index f776e2c..50d70ea 100644 --- a/windows.h +++ b/windows.h @@ -11,8 +11,7 @@

ifndef PYAHCORASICK_WINDOWS_H__

define PYAHCORASICK_WINDOWS_H__

-typedef unsigned char uint8_t; -typedef short unsigned int uint16_t; +#include

define PY_OBJECT_HEAD_INIT PyVarObject_HEAD_INIT(NULL, 0)

[WojciechMula]

@woakesd Thank you for the fix, but I'm not sure if stdint.h is present in earlier versions of Visual Studio and we stayed with custom typedefs.

What are the problems with tests?

[woakesd]

I did try running nmake test but that didn't get far (probably expected!).

If you want to avoid stdint.h (and I think you are correct about it not being in earlier versions of VS then some extra custom typedefs are needed in windows.h).

Having a busy weekend with wife and baby daughter ill. I'll put try sorting out the required defs on Monday!

[woakesd]

Have posted a pull request with the only missing typedef added

[WojciechMula]

@woakesd David, thank you for the fix, hope your child will get better quickly.

If you could find nice ifdefs, it would be great. I feel we should use stdint.h whenever it's possible.

[pombredanne]

@woakesd @WojciechMula would these do? https://github.com/chemeris/msinttypes/blob/master/stdint.h They are pretty considered the gold standard for this AFAIK.

[WojciechMula]

@pombredanne Thanks, that looks good. We should use it.

[woakesd]

I suggest using _MSC_VER to determine compiler version. I think we only need a substitute when compiling for Python 2.7.

[WojciechMula]

Why only Python2.7, what about Python 3.x?

[woakesd]

Correct me if I am wrong, but Python 3.x is compiled with Visual Studio 2010 which included stdint.h

Hmm, can you use a #ifdef to prevent the inclusion of a file and have it work when the file isn't present? Not sure...

[woakesd]

Simpler to use the substitute file I guess!

[woakesd]

Closing this as the patch fixes this issue

[pombredanne]

@woakesd Let me keep this open a bit so that @WojciechMula has a chance to review ...

[WojciechMula]

To be honest I'm not familiar with all these Windows things. It seems it's just a matter of proper typedefs, however I would opt for msinittypes (or similar one, as the lib seems to be unmaintained), to avoid reinventing ifdefs.

[pombredanne]

I have seen msinittypes used times and times again, ints are not something that change often ;)

[WojciechMula]

This looks like the perfect recommendation. :) So, we will use that lib.

[WojciechMula]

I included that library, looks OK.

[StSchulze]

Hi, unfortunately I am unable to install pyahocorasick on Windows 10 with pip:

C:\Users\Admin\Desktop>pip install pyahocorasick
Collecting pyahocorasick
  Using cached pyahocorasick-1.1.4.tar.bz2
Installing collected packages: pyahocorasick
  Running setup.py install for pyahocorasick ... error
    Complete output from command "c:\program files\python 3.5\python.exe" -u -c "import setuptools, tokenize;__file__='C:\\Users\\Admin\\AppData\\Local\\Temp\\pip-build-qc37f_a0\\pyahocorasick\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record C:\Users\Admin\AppData\Local\Temp\pip-3hvzfkaw-record\install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_ext
    building 'ahocorasick' extension
    error: [WinError 2] The system cannot find the file specified

    ----------------------------------------
Command ""c:\program files\python 3.5\python.exe" -u -c "import setuptools, tokenize;__file__='C:\\Users\\Admin\\AppData\\Local\\Temp\\pip-build-qc37f_a0\\pyahocorasick\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record C:\Users\Admin\AppData\Local\Temp\pip-3hvzfkaw-record\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in C:\Users\Admin\AppData\Local\Temp\pip-build-qc37f_a0\pyahocorasick\

I am not familiar with MSVC at all. Do you have an idea how to solve that problem?

[WojciechMula]

@StSchulze Do you have WinSDK installed? Which version?

[StSchulze]

I have installed WinSDK v10.0 now but I still get the same error.

Also this issue #https://github.com/WojciechMula/pyahocorasick/issues/55 did not really help me out since there does not seem to be a 'Windows SDK Command Prompt' anymore ...

[StSchulze]

I installed Visual Studio 2017 now and got a Developper Command Prompt at least. However, the isntallation using wheel does not work for me:

C:\Users\Admin\Desktop\pyahocorasick>python setup.py bdist_wheel upload
running bdist_wheel
running build
running build_ext
building 'ahocorasick' extension
creating build
creating build\temp.win-amd64-3.5
creating build\temp.win-amd64-3.5\Release
cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MT -DAHOCORASICK_UNICODE= "-IC:\Program Files\Python 3.5\include" "-IC:\Program Files\Python 3.5\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.15063.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.15063.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.15063.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.15063.0\winrt" /Tcpyahocorasick.c /Fobuild\temp.win-amd64-3.5\Release\pyahocorasick.obj
error: command 'cl.exe' failed: No such file or directory

[woakesd]

Try upgrading setup tools:

py -m pip install --upgrade setuptools

[StSchulze]

upgraded setuptools, but still the same error (error: command 'cl.exe' failed: No such file or directory)

[WojciechMula]

@StSchulze Did you run pip from Visual Studio Command Prompt? Such link should be available via Start menu.

[StSchulze]

@WojciechMula Yes, I run it from "Developer Command Prompt for VS 2017"

So when I just run pip install I get the following:

C:\Windows\System32>pip install pyahocorasick
Collecting pyahocorasick
  Using cached pyahocorasick-1.1.4.tar.bz2
Building wheels for collected packages: pyahocorasick
  Running setup.py bdist_wheel for pyahocorasick ... error
  Complete output from command "c:\program files\python 3.5\python.exe" -u -c "import setuptools, tokenize;__file__='C:\\Users\\Admin\\AppData\\Local\\Temp\\pip-build-kct1m5_j\\pyahocorasick\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d C:\Users\Admin\AppData\Local\Temp\tmprkwn1zpxpip-wheel- --python-tag cp35:
  running bdist_wheel
  running build
  running build_ext
  building 'ahocorasick' extension
  error: [WinError 2] The system cannot find the file specified

  ----------------------------------------
  Failed building wheel for pyahocorasick
  Running setup.py clean for pyahocorasick
Failed to build pyahocorasick
Installing collected packages: pyahocorasick
  Running setup.py install for pyahocorasick ... error
    Complete output from command "c:\program files\python 3.5\python.exe" -u -c "import setuptools, tokenize;__file__='C:\\Users\\Admin\\AppData\\Local\\Temp\\pip-build-kct1m5_j\\pyahocorasick\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record C:\Users\Admin\AppData\Local\Temp\pip-p3kc44co-record\install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_ext
    building 'ahocorasick' extension
    error: [WinError 2] The system cannot find the file specified

    ----------------------------------------
Command ""c:\program files\python 3.5\python.exe" -u -c "import setuptools, tokenize;__file__='C:\\Users\\Admin\\AppData\\Local\\Temp\\pip-build-kct1m5_j\\pyahocorasick\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record C:\Users\Admin\AppData\Local\Temp\pip-p3kc44co-record\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in C:\Users\Admin\AppData\Local\Temp\pip-build-kct1m5_j\pyahocorasick\

When I try to follow the instructions for building the wheel package set MSSDK=1 set DISTUTILS_USE_SDK=1 I get the following:

C:\Users\Admin\Desktop\pyahocorasick>python setup.py bdist_wheel upload
running bdist_wheel
running build
running build_ext
building 'ahocorasick' extension
cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MT -DAHOCORASICK_UNICODE= "-IC:\Program Files\Python 3.5\include" "-IC:\Program Files\Python 3.5\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.15063.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.15063.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.15063.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.15063.0\winrt" /Tcpyahocorasick.c /Fobuild\temp.win-amd64-3.5\Release\pyahocorasick.obj
error: command 'cl.exe' failed: No such file or directory

and for pip something similar:

C:\Users\Admin\Desktop\pyahocorasick>pip install pyahocorasick
Collecting pyahocorasick
  Using cached pyahocorasick-1.1.4.tar.bz2
Building wheels for collected packages: pyahocorasick
  Running setup.py bdist_wheel for pyahocorasick ... error
  Complete output from command "c:\program files\python 3.5\python.exe" -u -c "import setuptools, tokenize;__file__='C:\\Users\\Admin\\AppData\\Local\\Temp\\pip-build-_zopf_zd\\pyahocorasick\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d C:\Users\Admin\AppData\Local\Temp\tmp384f0yzwpip-wheel- --python-tag cp35:
  running bdist_wheel
  running build
  running build_ext
  building 'ahocorasick' extension
  creating build
  creating build\temp.win-amd64-3.5
  creating build\temp.win-amd64-3.5\Release
  cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MT -DAHOCORASICK_UNICODE= "-Ic:\program files\python 3.5\include" "-Ic:\program files\python 3.5\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.15063.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.15063.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.15063.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.15063.0\winrt" /Tcpyahocorasick.c /Fobuild\temp.win-amd64-3.5\Release\pyahocorasick.obj
  error: command 'cl.exe' failed: No such file or directory

  ----------------------------------------
  Failed building wheel for pyahocorasick
  Running setup.py clean for pyahocorasick
Failed to build pyahocorasick
Installing collected packages: pyahocorasick
  Running setup.py install for pyahocorasick ... error
    Complete output from command "c:\program files\python 3.5\python.exe" -u -c "import setuptools, tokenize;__file__='C:\\Users\\Admin\\AppData\\Local\\Temp\\pip-build-_zopf_zd\\pyahocorasick\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record C:\Users\Admin\AppData\Local\Temp\pip-84oj9epl-record\install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_ext
    building 'ahocorasick' extension
    creating build
    creating build\temp.win-amd64-3.5
    creating build\temp.win-amd64-3.5\Release
    cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MT -DAHOCORASICK_UNICODE= "-Ic:\program files\python 3.5\include" "-Ic:\program files\python 3.5\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.15063.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.15063.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.15063.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.15063.0\winrt" /Tcpyahocorasick.c /Fobuild\temp.win-amd64-3.5\Release\pyahocorasick.obj
    error: command 'cl.exe' failed: No such file or directory

    ----------------------------------------
Command ""c:\program files\python 3.5\python.exe" -u -c "import setuptools, tokenize;__file__='C:\\Users\\Admin\\AppData\\Local\\Temp\\pip-build-_zopf_zd\\pyahocorasick\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record C:\Users\Admin\AppData\Local\Temp\pip-84oj9epl-record\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in C:\Users\Admin\AppData\Local\Temp\pip-build-_zopf_zd\pyahocorasick\

[zhll912]

I've get the same issue,and up solutions can't solve it.And another method ?

[WayneKeenan]

did you ever find out why cl.exe can't be found? I get this for a project of mine when I use Python 3.5.2, but not when switching to 3.6.3

[StSchulze]

I could never solve the issue but am working on a different OS now, so I didn't follow up on this after a while.

However, maybe this helps you: https://github.com/ursgal/ursgal/issues/30

[WayneKeenan]

Thanks, I manged to resolve it by running :

"C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Auxiliary\Build\vcvars64.bat"

prior to running setup.py.

There is also a vcvars32.bat for 32bit python installs.

The main difference I noticed between 3.5 and 3.6 was that a lot of Include paths were missing in the 3.5 command. I think missing *.h files is the real underlying error and not that cl.exe can't be found, as my 3.5 and 3.6 had the same PATH settings, but only 3.6 'found' cl.exe.

I'm guessing Python 3.6 hunts the system/registry to find VC and its flags in a better way than Python 3.5 does.

[pombredanne]

Since 3.5 and 3.6 are all dead snakes.... I am inclined to close this now. Any objection?

[pombredanne]

I am closing this now. The upcoming 1.4.3 will have pre-built wheel for all OSes using cibuildwheel. The lowest supported version becomes Python 3.6.