pnlbwh / CNN-Diffusion-MRIBrain-Segmentation

CNN based brain masking
Other
14 stars 10 forks source link

gcc error from psutil install #20

Closed tashrifbillah closed 2 months ago

tashrifbillah commented 4 years ago

A gcc related error occurs in GPU machines grx03 and pnl-maxwell when psutil is attempted to install as part of conda environment inside - pip: block. They both have gcc -V:

gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-36)

Reason for this failure is unknown:

    creating build/temp.linux-x86_64-3.6/psutil
    /data/pnl/soft/pnlpipe3/miniconda3/envs/dmri_seg/bin/x86_64-conda_cos6-linux-gnu-cc -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /data/pnl/soft/pnlpipe3/miniconda3/envs/dmri_seg/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /data/pnl/soft/pnlpipe3/miniconda3/envs/dmri_seg/include -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=570 -DPSUTIL_LINUX=1 -I/data/pnl/soft/pnlpipe3/miniconda3/envs/dmri_seg/include/python3.6m -c psutil/_psutil_common.c -o build/temp.linux-x86_64-3.6/psutil/_psutil_common.o
    /data/pnl/soft/pnlpipe3/miniconda3/envs/dmri_seg/bin/x86_64-conda_cos6-linux-gnu-cc -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /data/pnl/soft/pnlpipe3/miniconda3/envs/dmri_seg/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /data/pnl/soft/pnlpipe3/miniconda3/envs/dmri_seg/include -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=570 -DPSUTIL_LINUX=1 -I/data/pnl/soft/pnlpipe3/miniconda3/envs/dmri_seg/include/python3.6m -c psutil/_psutil_posix.c -o build/temp.linux-x86_64-3.6/psutil/_psutil_posix.o
    /data/pnl/soft/pnlpipe3/miniconda3/envs/dmri_seg/bin/x86_64-conda_cos6-linux-gnu-cc -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /data/pnl/soft/pnlpipe3/miniconda3/envs/dmri_seg/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /data/pnl/soft/pnlpipe3/miniconda3/envs/dmri_seg/include -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=570 -DPSUTIL_LINUX=1 -I/data/pnl/soft/pnlpipe3/miniconda3/envs/dmri_seg/include/python3.6m -c psutil/_psutil_linux.c -o build/temp.linux-x86_64-3.6/psutil/_psutil_linux.o
    psutil/_psutil_linux.c: In function 'PyInit__psutil_linux':
    psutil/_psutil_linux.c:612:15: warning: unused variable 'v' [-Wunused-variable]
         PyObject *v;
                   ^
    gcc -pthread -shared -L/data/pnl/soft/pnlpipe3/miniconda3/envs/dmri_seg/lib -Wl,-rpath=/data/pnl/soft/pnlpipe3/miniconda3/envs/dmri_seg/lib,--no-as-needed -L/data/pnl/soft/pnlpipe3/miniconda3/envs/dmri_seg/lib -Wl,-rpath=/data/pnl/soft/pnlpipe3/miniconda3/envs/dmri_seg/lib,--no-as-needed -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,-rpath,/data/pnl/soft/pnlpipe3/miniconda3/envs/dmri_seg/lib -Wl,-rpath-link,/data/pnl/soft/pnlpipe3/miniconda3/envs/dmri_seg/lib -L/data/pnl/soft/pnlpipe3/miniconda3/envs/dmri_seg/lib -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /data/pnl/soft/pnlpipe3/miniconda3/envs/dmri_seg/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /data/pnl/soft/pnlpipe3/miniconda3/envs/dmri_seg/include build/temp.linux-x86_64-3.6/psutil/_psutil_common.o build/temp.linux-x86_64-3.6/psutil/_psutil_posix.o build/temp.linux-x86_64-3.6/psutil/_psutil_linux.o -L/data/pnl/soft/pnlpipe3/miniconda3/envs/dmri_seg/lib -lpython3.6m -o build/lib.linux-x86_64-3.6/psutil/_psutil_linux.cpython-36m-x86_64-linux-gnu.so
    gcc: error: unrecognized command line option ‘-fno-plt’
    error: command 'gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /data/pnl/soft/pnlpipe3/miniconda3/envs/dmri_seg/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-aykwzsnf/psutil/setup.py'"'"'; __file__='"'"'/tmp/pip-install-aykwzsnf/psutil/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-2yo77_bn/install-record.txt --single-version-externally-managed --compile --install-headers /data/pnl/soft/pnlpipe3/miniconda3/envs/dmri_seg/include/python3.6m/psutil Check the logs for full command output.

CondaEnvException: Pip failed

Updating conda didn't solve this problem.

The following is the environment_gpu.yml file:


name: dmri_seg

channels:
    - conda-forge
    - pnlbwh

dependencies:
    - python==3.6
    - tensorflow-gpu==1.12.0
    - cudatoolkit==9.0
    - cudnn==7.6.0
    - keras==2.2.4
    - numpy==1.16.4
    - nibabel>=2.2.1
    - ants
    - pip
    - pip:
        - gputil
        - scikit-image==0.16.2
        - git+https://github.com/pnlbwh/conversion.git

The conversion package has psutil as its dependency.

The error didn't occur in pnl-z840-2 which is not a GPU machine and have gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39). Notice -39 here is different from -35 above.

Installing psutil in the dependencies block first, solves the problem:


name: dmri_seg

channels:
    - conda-forge
    - pnlbwh

dependencies:
    - python==3.6
    - tensorflow-gpu==1.12.0
    - cudatoolkit==9.0
    - cudnn==7.6.0
    - keras==2.2.4
    - numpy==1.16.4
    - nibabel>=2.2.1
    - ants
    - psutil
    - pip
    - pip:
        - gputil
        - scikit-image==0.16.2
        - git+https://github.com/pnlbwh/conversion.git
tashrifbillah commented 2 months ago

Ryan significantly improved the installation infrastructure. This issue was not observed to occur after his improvement.