search

snipsco / snips-nlu

Snips Python library to extract meaning from text
https://snips-nlu.readthedocs.io
Apache License 2.0
3.89k stars 513 forks source link

`error: Can not find Rust compiler` on `armv7-unknown-linux-gnueabihf` #861

Open DamonBlais opened 4 years ago

DamonBlais commented 4 years ago

First, I ran the rust installer via rustup:

info: profile set to 'default'                                                                                                                                           
info: syncing channel updates for 'stable-armv7-unknown-linux-gnueabihf'                                                                                                 
info: latest update on 2019-11-07, rust version 1.39.0 (4560ea788 2019-11-04)                                                                                            
warning: Force-skipping unavailable component 'clippy-armv7-unknown-linux-gnueabihf'                                                                                     
warning: Force-skipping unavailable component 'rustfmt-armv7-unknown-linux-gnueabihf'                                                                                    
info: downloading component 'cargo'                                                                                                                                      
info: downloading component 'rust-std'                                                                                                                                   
182.3 MiB / 182.3 MiB (100 %)  11.2 MiB/s in 22s ETA:  0s                                                                                                                
info: downloading component 'rustc'                                                                                                                                      
 52.0 MiB /  52.0 MiB (100 %)  11.0 MiB/s in  9s ETA:  0s                                                                                                                
info: installing component 'cargo'                                                                                                                                       
  4.3 MiB /   4.3 MiB (100 %)   3.3 MiB/s in  1s ETA:  0s                                                                                                                
info: installing component 'rust-std'                                                                                                                                    
182.3 MiB / 182.3 MiB (100 %)   2.6 MiB/s in 32s ETA:  0s                                                                                                                
  3 iops /   3 iops (100 %)   0 iops/s in  4s ETA:  0s                                                                                                                   
info: installing component 'rustc'                                                                                                                                       
 52.0 MiB /  52.0 MiB (100 %)   2.7 MiB/s in 35s ETA:  0s                                                                                                                
info: default toolchain set to 'stable'                                                                                                                                  
                                                                                                                                                                           stable installed - rustc 1.39.0 (4560ea788 2019-11-04)                                                                                                                 

Rust is installed now. Great!

I then relogged (to fill my environment).

I then installed the other things required for compilation:

sudo apt update
sudo apt install cython cython3 python-setuptools python3-setuptools
sudo apt install python-virtualenv python3-virtualenv
sudo pip install setuptools_rust
sudo pip3 install setuptools_rust

I then tried to install snips-nlu

sudo pip3 install snips-nlu

Which results in it being unable to find the rust compiler, as required for multiple packages:

# ... snip ...
  running build_ext                                                                                                                                                      
  running build_rust                                                                                                                                                     
  error: Can not find Rust compiler                                                                                                                                      

  ----------------------------------------                                                                                                                               
  Failed building wheel for snips-nlu-parsers
# ... snip ...
  running build_ext
  running build_rust
  error: Can not find Rust compiler

  ----------------------------------------
  Failed building wheel for snips-nlu-utils

It took me hours to build numpy on this platform, and now I can't even get snips working :(

I clearly have the rust installer working, as I am able to compile the rust "Hello World" as it were:

user@v2:~$ rustc hello.rs 
user@v2:~$ ./hello
Hello World!
user@v2:~$ cat hello.rs 
fn main() {
    println!("Hello World!");
}
DamonBlais commented 4 years ago

More system information

user@v2:~$ uname -a
Linux v2 4.4.95-respeaker-r7 #2 SMP Thu Mar 15 17:43:15 CST 2018 armv7l GNU/Linux

user@v2:~$ python3 --version
Python 3.5.3

user@v2:~$ rustc --version
rustc 1.39.0 (4560ea788 2019-11-04)

user@v2:~$ cat /etc/issue
Debian GNU/Linux 9 \n \l

respeaker.io Debian Image 20180319
adrienball commented 4 years ago

Hi @DamonBlais , I think the sudo in sudo pip3 install snips-nlu may be causing the issue. Can you try running this command without the sudo? (If you are using virtualenv, you shouldn't have to use sudo) What does sudo rustc --version output ?

DamonBlais commented 4 years ago

Interesting.

sudo rustc --version confirmed that rust was not a system package (command not found).

I have re-run rustup as root (with sudo) now,

The build still fails:

user@v2:~/_app$ sudo pip3 install snips-nlu                                                                                                                                                          
Collecting snips-nlu                                                                                                                                                                                                
  Using cached https://files.pythonhosted.org/packages/89/bf/9a12efb71002b32b70575f9d16694d99edbba501a59caadb60876b1ca5de/snips_nlu-0.20.1-py2.py3-none-any.whl                                                     
Requirement already satisfied: num2words<0.6,>=0.5.6 in /usr/local/lib/python3.5/dist-packages (from snips-nlu)                                                                                                     
Collecting scikit-learn<0.22,>=0.21.1; python_version >= "3.5" (from snips-nlu)                                                                                                                                     
  Using cached https://files.pythonhosted.org/packages/1e/ce/9d8c88e68af0a5b5c5d78d8d2b7bcadfd45e1d6afc863ccb9aee30765b06/scikit-learn-0.21.3.tar.gz                                                                
Collecting snips-nlu-utils<0.10,>=0.9 (from snips-nlu)                                                                                                                                                              
  Using cached https://files.pythonhosted.org/packages/56/e1/66f9cd9dbd017bf0522ecd51c9b4010f3cfaf06058205bc5a57ee7e0d10d/snips_nlu_utils-0.9.1.tar.gz                                                                Ignoring pathlib: markers 'python_version < "3.4"' don't match your environment                                                                                                                                   Collecting snips-nlu-parsers<0.5,>=0.3.1 (from snips-nlu)                                                 
  Using cached https://files.pythonhosted.org/packages/1a/7f/f5a3de83b5995d4315d1f46deffcb38638bc47187270e6084a274de38e12/snips_nlu_parsers-0.4.2.tar.gz                                                            
  Ignoring pathlib: markers 'python_version < "3.4"' don't match your environment
Requirement already satisfied: sklearn-crfsuite<0.4,>=0.3.6 in /usr/local/lib/python3.5/dist-packages (from snips-nlu)
Requirement already satisfied: numpy<2.0,>=1.15 in /usr/local/lib/python3.5/dist-packages (from snips-nlu) 
Requirement already satisfied: future<0.18,>=0.16 in /usr/local/lib/python3.5/dist-packages (from snips-nlu)
Requirement already satisfied: deprecation<3.0,>=2.0 in /usr/local/lib/python3.5/dist-packages (from snips-nlu)
Requirement already satisfied: pyaml<20.0,>=17.0 in /usr/local/lib/python3.5/dist-packages (from snips-nlu)
Requirement already satisfied: scipy<2.0,>=1.0 in /usr/local/lib/python3.5/dist-packages (from snips-nlu)
Collecting requests<3.0,>=2.0 (from snips-nlu)
  Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Requirement already satisfied: docopt>=0.6.2 in /usr/local/lib/python3.5/dist-packages (from num2words<0.6,>=0.5.6->snips-nlu)
Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.5/dist-packages (from scikit-learn<0.22,>=0.21.1; python_version >= "3.5"->snips-nlu)
Requirement already satisfied: python-crfsuite>=0.8.3 in /usr/local/lib/python3.5/dist-packages (from sklearn-crfsuite<0.4,>=0.3.6->snips-nlu)
Requirement already satisfied: tabulate in /usr/local/lib/python3.5/dist-packages (from sklearn-crfsuite<0.4,>=0.3.6->snips-nlu)
Requirement already satisfied: six in /usr/lib/python3/dist-packages (from sklearn-crfsuite<0.4,>=0.3.6->snips-nlu)
Requirement already satisfied: tqdm>=2.0 in /usr/local/lib/python3.5/dist-packages (from sklearn-crfsuite<0.4,>=0.3.6->snips-nlu)
Requirement already satisfied: packaging in /usr/local/lib/python3.5/dist-packages (from deprecation<3.0,>=2.0->snips-nlu)
Requirement already satisfied: PyYAML in /usr/local/lib/python3.5/dist-packages (from pyaml<20.0,>=17.0->snips-nlu)
Collecting chardet<3.1.0,>=3.0.2 (from requests<3.0,>=2.0->snips-nlu)
  Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting certifi>=2017.4.17 (from requests<3.0,>=2.0->snips-nlu)
  Downloading https://files.pythonhosted.org/packages/b9/63/df50cac98ea0d5b006c55a399c3bf1db9da7b5a24de7890bc9cfd5dd9e99/certifi-2019.11.28-py2.py3-none-any.whl (156kB)
    100% |████████████████████████████████| 163kB 758kB/s 
Collecting idna<2.9,>=2.5 (from requests<3.0,>=2.0->snips-nlu)
  Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests<3.0,>=2.0->snips-nlu)
  Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Requirement already satisfied: pyparsing>=2.0.2 in /usr/local/lib/python3.5/dist-packages (from packaging->deprecation<3.0,>=2.0->snips-nlu)
Building wheels for collected packages: scikit-learn, snips-nlu-utils, snips-nlu-parsers
  Running setup.py bdist_wheel for scikit-learn ... -^[
done
  Stored in directory: /root/.cache/pip/wheels/3a/b7/0e/b9ca22eccbf7aa20db50726337f23fd6904fb25ae83e704563 
  Running setup.py bdist_wheel for snips-nlu-utils ... error
  Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-2xl9rz1d/snips-nlu-utils/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r
\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpssyt6bqzpip-wheel- --python-tag cp35:
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib
  creating build/lib/snips_nlu_utils
  copying snips_nlu_utils/string.py -> build/lib/snips_nlu_utils
  copying snips_nlu_utils/utils.py -> build/lib/snips_nlu_utils
  copying snips_nlu_utils/token.py -> build/lib/snips_nlu_utils
  copying snips_nlu_utils/__init__.py -> build/lib/snips_nlu_utils
  running egg_info
  writing top-level names to snips_nlu_utils.egg-info/top_level.txt
  writing dependency_links to snips_nlu_utils.egg-info/dependency_links.txt
  writing snips_nlu_utils.egg-info/PKG-INFO
  writing requirements to snips_nlu_utils.egg-info/requires.txt
  reading manifest file 'snips_nlu_utils.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  warning: no files found matching '*.so' under directory 'snips_nlu_utils'
  warning: no previously-included files matching '__pycache__' found anywhere in distribution
  warning: no previously-included files matching '*.py[cod]' found anywhere in distribution
  writing manifest file 'snips_nlu_utils.egg-info/SOURCES.txt'
  copying snips_nlu_utils/__version__ -> build/lib/snips_nlu_utils
  running build_ext
  running build_rust
  error: Can not find Rust compiler
  ----------------------------------------                                                                                                                                                                 

  Failed building wheel for snips-nlu-utils
  Running setup.py clean for snips-nlu-utils
  Running setup.py bdist_wheel for snips-nlu-parsers ... error
  Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-2xl9rz1d/snips-nlu-parsers/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('
\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpbojl4y6rpip-wheel- --python-tag cp35:
  /usr/lib/python3/dist-packages/setuptools/dist.py:333: UserWarning: Normalizing '0.4.2
  ' to '0.4.2'
    normalized_version,
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib
  creating build/lib/snips_nlu_parsers
  copying snips_nlu_parsers/utils.py -> build/lib/snips_nlu_parsers
  copying snips_nlu_parsers/__init__.py -> build/lib/snips_nlu_parsers
  copying snips_nlu_parsers/builtin_entity_parser.py -> build/lib/snips_nlu_parsers
  copying snips_nlu_parsers/builtin_entities.py -> build/lib/snips_nlu_parsers
  copying snips_nlu_parsers/gazetteer_entity_parser.py -> build/lib/snips_nlu_parsers
  running egg_info
  writing requirements to snips_nlu_parsers.egg-info/requires.txt
  writing dependency_links to snips_nlu_parsers.egg-info/dependency_links.txt
  writing snips_nlu_parsers.egg-info/PKG-INFO
  writing top-level names to snips_nlu_parsers.egg-info/top_level.txt
  reading manifest file 'snips_nlu_parsers.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  warning: no previously-included files matching '*' found under directory 'ffi/target/'
  warning: no previously-included files matching '__pycache__' found anywhere in distribution
  warning: no previously-included files matching '*.py[cod]' found anywhere in distribution
  writing manifest file 'snips_nlu_parsers.egg-info/SOURCES.txt'
  copying snips_nlu_parsers/__version__ -> build/lib/snips_nlu_parsers
  creating build/lib/snips_nlu_parsers/dylib
  copying snips_nlu_parsers/dylib/.gitignore -> build/lib/snips_nlu_parsers/dylib
  running build_ext
  running build_rust
  error: Can not find Rust compiler

  ----------------------------------------
  Failed building wheel for snips-nlu-parsers
  Running setup.py clean for snips-nlu-parsers
Successfully built scikit-learn
Failed to build snips-nlu-utils snips-nlu-parsers                                                                                                                                                          

Installing collected packages: scikit-learn, snips-nlu-utils, snips-nlu-parsers, chardet, certifi, idna, urllib3, requests, snips-nlu
  Running setup.py install for snips-nlu-utils ... error
    Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-2xl9rz1d/snips-nlu-utils/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('
\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-clpw58c2-record/install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-armv7l-3.5
    creating build/lib.linux-armv7l-3.5/snips_nlu_utils
    copying snips_nlu_utils/string.py -> build/lib.linux-armv7l-3.5/snips_nlu_utils
    copying snips_nlu_utils/utils.py -> build/lib.linux-armv7l-3.5/snips_nlu_utils
    copying snips_nlu_utils/token.py -> build/lib.linux-armv7l-3.5/snips_nlu_utils
    copying snips_nlu_utils/__init__.py -> build/lib.linux-armv7l-3.5/snips_nlu_utils
    running egg_info
    writing dependency_links to snips_nlu_utils.egg-info/dependency_links.txt
    writing snips_nlu_utils.egg-info/PKG-INFO
    writing requirements to snips_nlu_utils.egg-info/requires.txt
    writing top-level names to snips_nlu_utils.egg-info/top_level.txt
    reading manifest file 'snips_nlu_utils.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    warning: no files found matching '*.so' under directory 'snips_nlu_utils'
    warning: no previously-included files matching '__pycache__' found anywhere in distribution
    warning: no previously-included files matching '*.py[cod]' found anywhere in distribution
    writing manifest file 'snips_nlu_utils.egg-info/SOURCES.txt'
    copying snips_nlu_utils/__version__ -> build/lib.linux-armv7l-3.5/snips_nlu_utils
    running build_ext
    running build_rust
    error: Can not find Rust compiler

    ----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-2xl9rz1d/snips-nlu-utils/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();e
xec(compile(code, __file__, 'exec'))" install --record /tmp/pip-clpw58c2-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-2xl9rz1d/snips-nlu-util
s/

I need the package to be available to an application running as root, so a user-local installation won't suffice here. Note that I'm not using a virtualenv, but that python-virtualenv was a dependancy of another portion of the application, I apologize for adding that confusion.

Attempts to compile "for my user" pip3 install -U snips-nlu have resulted in hours of pip compiling scipy again... much slower for my user than as root (no idea why.)

maxbachmann commented 4 years ago

@DamonBlais it should work when you add it to your /root/.bashrc

sudo su
echo 'export PATH="$HOME/.cargo/bin:$PATH"' >> /root/.bashrc
source /root/.bashrc
pip3 install snips-nlu
piratos commented 4 years ago

Guess you ignored the last line of the rust setup script To configure your current shell run source $HOME/.cargo/env

ejkitchen commented 3 years ago

I encountered this same issue ("Cannot find Rust compiler") during a Docker build on an Apple Silicon M1 . The FROM untuntu command was pulling an ARM64 version of Ubuntu and pip install transformers was failing on " running build_rust" with the same error "error: Can not find Rust compiler ". So just before the line pip install transformers I added this

curl https://sh.rustup.rs -sSf | sh export PATH=$PATH:~/.cargo/bin

This installs the rustc compiler to /root/.cargo/bin and then when you run pip install transformers, you no longer get the rust compiler not found error.

nick2525 commented 2 years ago

Thanks, curl https://sh.rustup.rs -sSf | sh export PATH=$PATH:~/.cargo/bin helped to install cryptography for "home assistant"

vikas94 commented 2 years ago

@ejkitchen thanks for the solution, worked for me too.

zoldaten commented 1 year ago 
curl https://sh.rustup.rs -sSf | sh -s -- --default-toolchain nightly
export PATH="$HOME/.cargo/bin:$PATH"