Error while installing FAIRSEQ using pip install --editable ./

[RavneetDTU]

🐛 Bug

Error while installing FAIRSEQ.

To Reproduce

1. CMD: pip3 install --editable ./
2. Console Error:

Obtaining file:///media/ravneet/Work%20SSD/fairseq_repo Installing build dependencies ... done Getting requirements to build wheel ... done Installing backend dependencies ... done Preparing wheel metadata ... done Requirement already satisfied: numpy in /home/ravneet/.local/lib/python3.8/site-packages (from fairseq==1.4.0) (1.18.4) Requirement already satisfied: regex in /home/ravneet/.local/lib/python3.8/site-packages (from fairseq==1.4.0) (2020.5.14) Requirement already satisfied: cython in /home/ravneet/.local/lib/python3.8/site-packages (from fairseq==1.4.0) (0.29.19) Requirement already satisfied: cffi in /home/ravneet/.local/lib/python3.8/site-packages (from fairseq==1.4.0) (1.14.0) Requirement already satisfied: editdistance in /home/ravneet/.local/lib/python3.8/site-packages (from fairseq==1.4.0) (0.5.3) Requirement already satisfied: torch in /home/ravneet/.local/lib/python3.8/site-packages (from fairseq==1.4.0) (1.5.0) Requirement already satisfied: sacrebleu in /home/ravneet/.local/lib/python3.8/site-packages (from fairseq==1.4.0) (1.4.9) Requirement already satisfied: tqdm in /home/ravneet/.local/lib/python3.8/site-packages (from fairseq==1.4.0) (4.30.0) Requirement already satisfied: pycparser in /home/ravneet/.local/lib/python3.8/site-packages (from cffi->fairseq==1.4.0) (2.20) Requirement already satisfied: future in /usr/lib/python3/dist-packages (from torch->fairseq==1.4.0) (0.18.2) Requirement already satisfied: typing in /home/ravneet/.local/lib/python3.8/site-packages (from sacrebleu->fairseq==1.4.0) (3.7.4.1) Requirement already satisfied: portalocker in /home/ravneet/.local/lib/python3.8/site-packages (from sacrebleu->fairseq==1.4.0) (1.7.0) Installing collected packages: fairseq Attempting uninstall: fairseq Found existing installation: fairseq 0.9.0 Uninstalling fairseq-0.9.0: Successfully uninstalled fairseq-0.9.0 Running setup.py develop for fairseq ERROR: Command errored out with exit status 1: command: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/media/ravneet/Work SSD/fairseq_repo/setup.py'"'"'; file='"'"'/media/ravneet/Work SSD/fairseq_repo/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' develop --no-deps --user --prefix= cwd: /media/ravneet/Work SSD/fairseq_repo/ Complete output (28 lines): running develop WARNING: The user site-packages directory is disabled. error: can't create or remove files in install directory

The following error occurred while trying to add or remove files in the
installation directory:

    [Errno 13] Permission denied: '/usr/local/lib/python3.8/dist-packages/test-easy-install-10667.write-test'

The installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:

    /usr/local/lib/python3.8/dist-packages/

Perhaps your account does not have write access to this directory?  If the
installation directory is a system-owned directory, you may need to sign in
as the administrator or "root" account.  If you do not have administrative
access to this machine, you may wish to choose a different installation
directory, preferably one that is listed in your PYTHONPATH environment
variable.

For information on other options, you may wish to consult the
documentation at:

  https://setuptools.readthedocs.io/en/latest/easy_install.html

Please make the appropriate changes for your system and try again.

----------------------------------------

Rolling back uninstall of fairseq Moving to /home/ravneet/.local/bin/fairseq-eval-lm from /tmp/pip-uninstall-x8rz1yg1/fairseq-eval-lm Moving to /home/ravneet/.local/bin/fairseq-generate from /tmp/pip-uninstall-x8rz1yg1/fairseq-generate Moving to /home/ravneet/.local/bin/fairseq-interactive from /tmp/pip-uninstall-x8rz1yg1/fairseq-interactive Moving to /home/ravneet/.local/bin/fairseq-preprocess from /tmp/pip-uninstall-x8rz1yg1/fairseq-preprocess Moving to /home/ravneet/.local/bin/fairseq-score from /tmp/pip-uninstall-x8rz1yg1/fairseq-score Moving to /home/ravneet/.local/bin/fairseq-train from /tmp/pip-uninstall-x8rz1yg1/fairseq-train Moving to /home/ravneet/.local/bin/fairseq-validate from /tmp/pip-uninstall-x8rz1yg1/fairseq-validate Moving to /home/ravneet/.local/lib/python3.8/site-packages/examples/ from /home/ravneet/.local/lib/python3.8/site-packages/~xamples Moving to /home/ravneet/.local/lib/python3.8/site-packages/fairseq-0.9.0.dist-info/ from /home/ravneet/.local/lib/python3.8/site-packages/~airseq-0.9.0.dist-info Moving to /home/ravneet/.local/lib/python3.8/site-packages/fairseq/ from /home/ravneet/.local/lib/python3.8/site-packages/~airseq Moving to /home/ravneet/.local/lib/python3.8/site-packages/fairseq_cli/ from /home/ravneet/.local/lib/python3.8/site-packages/~airseq_cli ERROR: Command errored out with exit status 1: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/media/ravneet/Work SSD/fairseq_repo/setup.py'"'"'; file='"'"'/media/ravneet/Work SSD/fairseq_repo/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' develop --no-deps --user --prefix= Check the logs for full command output.

Expected behavior

Environment

• PyTorch Version: 1.6
• OS: Ubuntu 20.04 LTS
• Python version: 3.8.2

[pratos]

Even I'm facing the same issue while installing fairseq via poetry

$ airflow git:(search_fixes) ✗ poetry add git+https://github.com/pytorch/fairseq.git

Installing dependencies from lock file
Warning: The lock file is not up to date with the latest changes in pyproject.toml. You may be getting outdated dependencies. Run update to
 update them.

[SolverProblemError]
Because no versions of fairseq match !=0.9.0
 and fairseq (0.9.0) depends on editdistance (*), every version of fairseq requires editdistance (*).
So, because no versions of editdistance match *
 and dags depends on fairseq (*), version solving failed.

Tried to set it to a commit: #54baa2e72d1228ca7d40b78f676ed782285ae6c7

$ airflow git:(search_fixes) ✗ poetry add git+https://github.com/pytorch/fairseq.git#54baa2e72d1228ca7d40b78f676ed782285ae6c7

Updating dependencies
Resolving dependencies... (18.4s)
Resolving dependencies... (38.1s)

Writing lock file

Package operations: 1 install, 1 update, 1 removal

  - Installing editdistance (0.5.3)
  - Updating fairseq (0.9.0 ffecb4e -> 0.9.0 54baa2e)
  - Removing decorator (4.4.2)

Was able to successfully do it, if anyone wants to have a temporary work around.

[alexeib]

have you tried creating a virtualenv or conda environment and installing there? looks like you have permission issues

[marija-stanojevic]

Hello,

I am having a similar error. I am executing the command in a virtual environment. I am trying to run it on university HPC, so I don't have sudo access to the system. Is there a workaround for this?

    ERROR: Command errored out with exit status 1:
     command: /VIRTUAL_ENV_PATH/bin/python3.6 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/PATHfairseq/setup.py'"'"'; __file__='"'"'/PATH/fairseq/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps --user --prefix=
         cwd: /PATH/fairseq/
    Complete output (29 lines):
    running develop
    WARNING: The user site-packages directory is disabled.
    Checking .pth file support in /usr/local/lib64/python3.6/site-packages/
    error: can't create or remove files in install directory

    The following error occurred while trying to add or remove files in the
    installation directory:

        [Errno 13] Permission denied: '/usr/local/lib64/python3.6/site-packages/test-easy-install-28561.pth'

    The installation directory you specified (via --install-dir, --prefix, or
    the distutils default setting) was:

        /usr/local/lib64/python3.6/site-packages/

    Perhaps your account does not have write access to this directory?  If the
    installation directory is a system-owned directory, you may need to sign in
    as the administrator or "root" account.  If you do not have administrative
    access to this machine, you may wish to choose a different installation
    directory, preferably one that is listed in your PYTHONPATH environment
    variable.

    For information on other options, you may wish to consult the
    documentation at:

      https://setuptools.readthedocs.io/en/latest/easy_install.html

    Please make the appropriate changes for your system and try again.

    ----------------------------------------
ERROR: Command errored out with exit status 1: /VIRTUAL_ENV_PATH/python3.6 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/PATH/fairseq/setup.py'"'"'; __file__='"'"'/PATH/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps --user --prefix= Check the logs for full command output.

Environment OS: CentOS Linux 7 Python version: 3.6.2 Pip version 20.2.3

[mjpost]

I don't like doing pip install --editable ., because you can only have one instance of fairseq installed this way. If you have multiple fairseq projects going, you then need to manage multiple environments, which are likely entirely redundant, and switch between them. A better approach is to build fairseq like this:

python setup.py build develop

and then run it like this:

PYTHONPATH=/path/to/this/fairseq python -m fairseq_cli.train

[shuoyangd]

I don't like doing pip install --editable ., because you can only have one instance of fairseq installed this way. If you have multiple fairseq projects going, you then need to manage multiple environments, which are likely entirely redundant, and switch between them. A better approach is to build fairseq like this:

python setup.py build develop

and then run it like this:

PYTHONPATH=/path/to/this/fairseq python -m fairseq_cli.train

Second. Having to run pip install --editable . is a major hassle when having multiple copies of fairseq laying around to try different ideas, and not everyone is willing to have a dedicated virtual environment for each project.

[anujsaraswat741]

use pip install --src=/path/to/installation/directory --editable ./ , using --target won't work here

[yyyyxie]

I am facing the same problem. Anyone solve it?

[jung-youjin]

Hi everyone! Is there a solution to this issue? I'm currently facing the same problem as well :(

[stale[bot]]

This issue has been automatically marked as stale. If this issue is still affecting you, please leave any comment (for example, "bump"), and we'll keep it open. We are sorry that we haven't been able to prioritize it yet. If you have any new additional information, please include it with your comment!

[stale[bot]]

Closing this issue after a prolonged period of inactivity. If this issue is still present in the latest release, please create a new issue with up-to-date information. Thank you!