search

kpu / kenlm

KenLM: Faster and Smaller Language Model Queries
http://kheafield.com/code/kenlm/
Other
2.48k stars 510 forks source link

Cygwin: 'gcc' compilation fails at strerror_r #217

Open lulamzamo opened 5 years ago

lulamzamo commented 5 years ago

Hi,

I'm strugging with installing python:kenlm on Cygwin. Making lmplz, etc went fine. I'm a new to nix and could use any direction. The dump is as follow: $ pip3 install https://github.com/kpu/kenlm/archive/master.zip Collecting https://github.com/kpu/kenlm/archive/master.zip Downloading https://github.com/kpu/kenlm/archive/master.zip / 5.2MB 580kB/s Building wheels for collected packages: kenlm Building wheel for kenlm (setup.py) ... error ERROR: Complete output from command /usr/bin/python3.6 -u -c 'import setuptools, tokenize;file='"'"'/tmp/pip-req-build-u0t32c_g/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-ne8e7rxo --python-tag cp36: ERROR: running bdist_wheel running build running build_ext building 'kenlm' extension creating build creating build/temp.cygwin-3.0.7-x86_64-3.6 creating build/temp.cygwin-3.0.7-x86_64-3.6/util creating build/temp.cygwin-3.0.7-x86_64-3.6/lm creating build/temp.cygwin-3.0.7-x86_64-3.6/util/double-conversion creating build/temp.cygwin-3.0.7-x86_64-3.6/python gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -ggdb -O2 -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fstack-protector-strong --param=ssp-buffer-size=4 -fdebug-prefix-map=/usr/src/ports/python36/python36-3.6.8-1.x86_64/build=/usr/src/debug/python36-3.6.8-1 -fdebug-prefix-map=/usr/src/ports/python36/python36-3.6.8-1.x86_64/src/Python-3.6.8=/usr/src/debug/python36-3.6.8-1 -ggdb -O2 -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fstack-protector-strong --param=ssp-buffer-size=4 -fdebug-prefix-map=/usr/src/ports/python36/python36-3.6.8-1.x86_64/build=/usr/src/debug/python36-3.6.8-1 -fdebug-prefix-map=/usr/src/ports/python36/python36-3.6.8-1.x86_64/src/Python-3.6.8=/usr/src/debug/python36-3.6.8-1 -I. -I/usr/include/python3.6m -c util/bit_packing.cc -o build/temp.cygwin-3.0.7-x86_64-3.6/util/bit_packing.o -O3 -DNDEBUG -DKENLM_MAX_ORDER=6 -std=c++11 -DHAVE_ZLIB -DHAVE_BZLIB -DHAVE_XZLIB gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -ggdb -O2 -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fstack-protector-strong --param=ssp-buffer-size=4 -fdebug-prefix-map=/usr/src/ports/python36/python36-3.6.8-1.x86_64/build=/usr/src/debug/python36-3.6.8-1 -fdebug-prefix-map=/usr/src/ports/python36/python36-3.6.8-1.x86_64/src/Python-3.6.8=/usr/src/debug/python36-3.6.8-1 -ggdb -O2 -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fstack-protector-strong --param=ssp-buffer-size=4 -fdebug-prefix-map=/usr/src/ports/python36/python36-3.6.8-1.x86_64/build=/usr/src/debug/python36-3.6.8-1 -fdebug-prefix-map=/usr/src/ports/python36/python36-3.6.8-1.x86_64/src/Python-3.6.8=/usr/src/debug/python36-3.6.8-1 -I. -I/usr/include/python3.6m -c util/ersatz_progress.cc -o build/temp.cygwin-3.0.7-x86_64-3.6/util/ersatz_progress.o -O3 -DNDEBUG -DKENLM_MAX_ORDER=6 -std=c++11 -DHAVE_ZLIB -DHAVE_BZLIB -DHAVE_XZLIB .. .. util/exception.cc: In constructor ‘util::ErrnoException::ErrnoException()’: util/exception.cc:78:36: error: ‘strerror_r’ was not declared in this scope const char add = HandleStrerror(strerror_r(errno, buf, 200), buf); ^~~~~~ util/exception.cc:78:36: note: suggested alternative: ‘_strerror_r’ const char *add = HandleStrerror(strerror_r(errno, buf, 200), buf); ^~~~~~ _strerror_r error: command 'gcc' failed with exit status 1

ERROR: Command "/usr/bin/python3.6 -u -c 'import setuptools, tokenize;file='"'"'/tmp/pip-req-build-u0t32c_g/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-dz291i1j/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-req-build-u0t32c_g/

kpu commented 5 years ago

You're running windows. I don't support windows but welcome pull requests from contributors like you. It appears gcc is telling you what edit to make, in this case replacing strerror_r with _strerror_r.

anetschka commented 5 years ago

@lulamzamo I have tried various things on Windows 10. Here are the options that I know about:

  1. You could try using cmake from the Visual Studio developer command prompt. This hasn't worked for me, since I have VS 2019 installed and the VS files in the Windows folder require VS 2010 (!!) build tools and, it seems, VS 2010.
  2. You could compile from the Ubuntu subsystem. I haven't completely verified this option yet, but it seems to work. You can then use your trained model from the Python library that is included in this repository.
kpu commented 5 years ago

I would of course welcome a commit using a newer version of visual studio.

anetschka commented 5 years ago

@kpu not sure I have the time and/or expertise for this, but if I have, I will try.

anetschka commented 5 years ago

@kpu I propose to add the following bit of documentation for Windows users to the Readme:

Windows 10 users can use the Linux subsystem to compile kenlm:

  1. If you have not yet activated the Linux subsystem on Windows, follow the documentation to do so: https://docs.microsoft.com/en-us/windows/wsl/install-win10
  2. Open the comand prompt of your new Linux system from the Windows start menu (e.g. by typing "Ubuntu" if you have installed Ubuntu).
  3. In the Linux command prompt, navigate to the root directory. You can then access the Windows file system from the "mnt" folder. command
  4. Navigate to the directory that contains the kenlm code from GitHub. Follow the installation instructions for Linux to build.
  5. Use the Linux command prompt to train your models.
  6. Use your models from Windows through the kenlm Python library.