Pip Package not building in coiled software environment

[avriiil]

When building a software environment using a .yml file containing the pip package camel-tools, the Jupyter Notebook cell completes but the software environment isn't actually built because of the error (partially) copied below. On Slack channel @necaris suggests that this is because camel-tools requires camel-kenlm and the latter doesn't have binary wheels. Note that camel-tools is only available through pip and not supported by conda.

The issue is two-fold:

1. support for building non-conda packages that do not have binary wheels
2. the fact that the cell completes without a clear error message

Partial error message including the relevant mentions of camel-tools and camel-kenlm copied below:

Building wheels for collected packages: camel-tools, camel-kenlm, docopt, sacremoses
  Building wheel for camel-tools (setup.py): started
  Building wheel for camel-tools (setup.py): finished with status 'done'
  Created wheel for camel-tools: filename=camel_tools-1.1.0-py3-none-any.whl size=96894 sha256=9d905266c35c0e7f0a96a91eb4be690d7df92725cd50b2ca253664e1c1229860
  Stored in directory: /root/.cache/pip/wheels/7f/9f/4a/43286ef26748f0831defc8f590926d30c22b210853e1e4af89
  Building wheel for camel-kenlm (setup.py): started
  Building wheel for camel-kenlm (setup.py): finished with status 'error'
  Running setup.py clean for camel-kenlm
  Building wheel for docopt (setup.py): started
  Building wheel for docopt (setup.py): finished with status 'done'
  Created wheel for docopt: filename=docopt-0.6.2-py2.py3-none-any.whl size=13705 sha256=bc1ec9af3e881822efd962959a7071e27a6cb4e834b060c01c65b4f3821804d5
  Stored in directory: /root/.cache/pip/wheels/72/b0/3f/1d95f96ff986c7dfffe46ce2be4062f38ebd04b506c77c81b9
  Building wheel for sacremoses (setup.py): started
  Building wheel for sacremoses (setup.py): finished with status 'done'
  Created wheel for sacremoses: filename=sacremoses-0.0.43-py3-none-any.whl size=893258 sha256=27a9a5fd38b38ba9a95d1985d8dc0004c482b0f12680aa04d83db323325f04c5
  Stored in directory: /root/.cache/pip/wheels/69/09/d1/bf058f7d6fa0ecba2ce7c66be3b8d012beb4bf61a6e0c101c0
Successfully built camel-tools docopt sacremoses
Failed to build camel-kenlm
Installing collected packages: tqdm, regex, joblib, tokenizers, threadpoolctl, sentencepiece, sacremoses, filelock, transformers, torch, scikit-learn, editdistance, docopt, dill, camel-kenlm, camel-tools
    Running setup.py install for camel-kenlm: started
    Running setup.py install for camel-kenlm: finished with status 'error'
ERROR conda.cli.main_run:execute(33): Subprocess for 'conda run ['/bin/bash', '-c', 'pip install -r requirements.txt     && rm requirements.txt']' command failed.  (See above for error)
  ERROR: Command errored out with exit status 1:
   command: /opt/conda/envs/coiled/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-dh9yiftq/camel-kenlm_c89ab8736825404ca0bcb2f6f5a17709/setup.py'"'"'; __file__='"'"'/tmp/pip-install-dh9yiftq/camel-kenlm_c89ab8736825404ca0bcb2f6f5a17709/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-to_fvw3v
       cwd: /tmp/pip-install-dh9yiftq/camel-kenlm_c89ab8736825404ca0bcb2f6f5a17709/
  Complete output (13 lines):
  running bdist_wheel
  running build
  running build_ext
  building 'kenlm' extension
  creating build
  creating build/temp.linux-x86_64-3.7
  creating build/temp.linux-x86_64-3.7/util
  creating build/temp.linux-x86_64-3.7/lm
  creating build/temp.linux-x86_64-3.7/util/double-conversion
  creating build/temp.linux-x86_64-3.7/python
  gcc -pthread -B /opt/conda/envs/coiled/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I. -I/opt/conda/envs/coiled/include/python3.7m -c util/bit_packing.cc -o build/temp.linux-x86_64-3.7/util/bit_packing.o -O3 -DNDEBUG -DKENLM_MAX_ORDER=6 -std=c++11
  gcc: error trying to exec 'cc1plus': execvp: No such file or directory
  error: command 'gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for camel-kenlm
    ERROR: Command errored out with exit status 1:
     command: /opt/conda/envs/coiled/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-dh9yiftq/camel-kenlm_c89ab8736825404ca0bcb2f6f5a17709/setup.py'"'"'; __file__='"'"'/tmp/pip-install-dh9yiftq/camel-kenlm_c89ab8736825404ca0bcb2f6f5a17709/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-i75tgh8u/install-record.txt --single-version-externally-managed --compile --install-headers /opt/conda/envs/coiled/include/python3.7m/camel-kenlm
         cwd: /tmp/pip-install-dh9yiftq/camel-kenlm_c89ab8736825404ca0bcb2f6f5a17709/
    Complete output (13 lines):
    running install
    running build
    running build_ext
    building 'kenlm' extension
    creating build
    creating build/temp.linux-x86_64-3.7
    creating build/temp.linux-x86_64-3.7/util
    creating build/temp.linux-x86_64-3.7/lm
    creating build/temp.linux-x86_64-3.7/util/double-conversion
    creating build/temp.linux-x86_64-3.7/python
    gcc -pthread -B /opt/conda/envs/coiled/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I. -I/opt/conda/envs/coiled/include/python3.7m -c util/bit_packing.cc -o build/temp.linux-x86_64-3.7/util/bit_packing.o -O3 -DNDEBUG -DKENLM_MAX_ORDER=6 -std=c++11
    gcc: error trying to exec 'cc1plus': execvp: No such file or directory
    error: command 'gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /opt/conda/envs/coiled/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-dh9yiftq/camel-kenlm_c89ab8736825404ca0bcb2f6f5a17709/setup.py'"'"'; __file__='"'"'/tmp/pip-install-dh9yiftq/camel-kenlm_c89ab8736825404ca0bcb2f6f5a17709/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-i75tgh8u/install-record.txt --single-version-externally-managed --compile --install-headers /opt/conda/envs/coiled/include/python3.7m/camel-kenlm Check the logs for full command output.
subprocess exited with status 1
subprocess exited with status 1
error building at STEP "RUN pip install -r requirements.txt     && rm requirements.txt": exit status 1

[ian-r-rose]

Hi @rrpelgrim, thanks for your patience. A fix for this is in progress, and we should be able to deploy it soon

[avriiil]

Excellent, that would be a huge help! Do you think that will be in the next few days? Just trying to see whether I should go ahead and process the data locally (with loooong processing times) or wait until I can use my coiled cluster. thanks again!

-- Richard Pelgrim +31 (0)6 374 807 31

On Wed, Apr 7, 2021 at 8:32 AM Ian Rose @.***> wrote:

Hi @rrpelgrim https://github.com/rrpelgrim, thanks for your patience. A fix for this is in progress, and we should be able to deploy it soon

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/coiled/feedback/issues/140#issuecomment-815011130, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQLWMSXN5GYDVEXKV4MAHVTTHR3KNANCNFSM42JTK7TA .

[necaris]

We're expecting to roll this out to production in the next couple of days, yep :-)

On Wed, Apr 7, 2021 at 3:17 PM Richard Pelgrim @.***> wrote:

Excellent, that would be a huge help! Do you think that will be in the next few days? Just trying to see whether I should go ahead and process the data locally (with loooong processing times) or wait until I can use my coiled cluster. thanks again!

-- Richard Pelgrim +31 (0)6 374 807 31

On Wed, Apr 7, 2021 at 8:32 AM Ian Rose @.***> wrote:

Hi @rrpelgrim https://github.com/rrpelgrim, thanks for your patience. A fix for this is in progress, and we should be able to deploy it soon

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/coiled/feedback/issues/140#issuecomment-815011130, or unsubscribe < https://github.com/notifications/unsubscribe-auth/AQLWMSXN5GYDVEXKV4MAHVTTHR3KNANCNFSM42JTK7TA

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/coiled/feedback/issues/140#issuecomment-815162027, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADQPYMIXXNXR7A5EASZPTLTHSVURANCNFSM42JTK7TA .

-- Rami Chowdhury coiled.io "A mind all logic is like a knife all blade -- it makes the hand bleed that uses it." -- Rabindranath Tagore

[avriiil]

Hi @necaris @ian-r-rose, just wondering if there's any update on this issue?

[ian-r-rose]

Hi @rrpelgrim, sorry for the slow reply. This fix should be deployed now: I was just able to create a new software environment with

import coiled
cloud = coiled.Cloud()
cloud.create_software_environment("pip-build-env", pip=["camel-kenlm"])

Please let us know if you run into any more problems around this!

[avriiil]

Fantastic, thanks!

[ian-r-rose]

Closing, but if you run into any more troubles, feel free to ping here and reopen!

[avriiil]

Just confirming that this is now working on my end as well. Thanks again!