search

UBC-Stat-ML / fitclone

14 stars 3 forks source link

Installation issue (gcc error) #12

Open vleblanc opened 2 years ago

vleblanc commented 2 years ago

Hello! I'm running into some issues with installation, and I'm hoping you can help. I've narrowed it down to a problem in the cyconvert.sh script, where I get the same error on every line:

(fitclone) [vleblanc@marrahost02 fitclone]$ python3 setup_cython/setup.py build_ext --inplace
Compiling wf_sample_parallel.pyx because it changed.
[1/1] Cythonizing wf_sample_parallel.pyx
/projects/vleblanc_prj/tools/anaconda3/envs/fitclone/lib/python3.7/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /projects/vleblanc_prj/tools/fitclone/fitclone/wf_sample_parallel.pyx
  tree = Parsing.p_module(s, pxd, full_module_name)
running build_ext
building 'wf_sample_parallel' extension
creating build
creating build/temp.linux-x86_64-3.7
gcc -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/projects/vleblanc_prj/tools/anaconda3/envs/fitclone/lib/python3.7/site-packages/numpy/core/include -I/projects/vleblanc_prj/tools/anaconda3/envs/fitclone/include/python3.7m -c wf_sample_parallel.c -o build/temp.linux-x86_64-3.7/wf_sample_parallel.o -fopenmp
In file included from /projects/vleblanc_prj/tools/anaconda3/envs/fitclone/lib/python3.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1822,
                 from /projects/vleblanc_prj/tools/anaconda3/envs/fitclone/lib/python3.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
                 from /projects/vleblanc_prj/tools/anaconda3/envs/fitclone/lib/python3.7/site-packages/numpy/core/include/numpy/arrayobject.h:4,
                 from wf_sample_parallel.c:629:
/projects/vleblanc_prj/tools/anaconda3/envs/fitclone/lib/python3.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
   17 | #warning "Using deprecated NumPy API, disable it with " \
      |  ^~~~~~~
gcc -pthread -shared -B /projects/vleblanc_prj/tools/anaconda3/envs/fitclone/compiler_compat -L/projects/vleblanc_prj/tools/anaconda3/envs/fitclone/lib -Wl,-rpath=/projects/vleblanc_prj/tools/anaconda3/envs/fitclone/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.7/wf_sample_parallel.o -o /projects/vleblanc_prj/tools/fitclone/fitclone/wf_sample_parallel.cpython-37m-x86_64-linux-gnu.so -fopenmp
/projects/vleblanc_prj/tools/anaconda3/envs/fitclone/compiler_compat/ld: warning: /projects/vleblanc_prj/tools/anaconda3/envs/fitclone/bin/../lib/gcc/x86_64-conda-linux-gnu/11.2.0/crtbeginS.o: unsupported GNU_PROPERTY_TYPE (5) type: 0xc0010002
/projects/vleblanc_prj/tools/anaconda3/envs/fitclone/compiler_compat/ld: warning: /projects/vleblanc_prj/tools/anaconda3/envs/fitclone/bin/../lib/gcc/x86_64-conda-linux-gnu/11.2.0/crtbeginS.o: unsupported GNU_PROPERTY_TYPE (5) type: 0xc0010001
/projects/vleblanc_prj/tools/anaconda3/envs/fitclone/compiler_compat/ld: build/temp.linux-x86_64-3.7/wf_sample_parallel.o: unable to initialize decompress status for section .debug_info
/projects/vleblanc_prj/tools/anaconda3/envs/fitclone/compiler_compat/ld: build/temp.linux-x86_64-3.7/wf_sample_parallel.o: unable to initialize decompress status for section .debug_info
/projects/vleblanc_prj/tools/anaconda3/envs/fitclone/compiler_compat/ld: build/temp.linux-x86_64-3.7/wf_sample_parallel.o: unable to initialize decompress status for section .debug_info
/projects/vleblanc_prj/tools/anaconda3/envs/fitclone/compiler_compat/ld: build/temp.linux-x86_64-3.7/wf_sample_parallel.o: unable to initialize decompress status for section .debug_info
build/temp.linux-x86_64-3.7/wf_sample_parallel.o: file not recognized: file format not recognized
collect2: error: ld returned 1 exit status
error: command 'gcc' failed with exit status 1

I've tried to hunt down these errors on various forums and it seems that it may be an issue of using a different version of the compiler that was used to compile the files originally. I don't think the cython language level or the deprecated NumPy API are the issue, but perhaps I'm wrong (and please let me know if I should also address these). Could I please get your help in solving this? I'd love to try out this tool :)

In case it's helpful, here's everything installed in my fitclone conda environment:

(fitclone) [vleblanc@marrahost02 fitclone]$ conda list
# packages in environment at /projects/vleblanc_prj/tools/anaconda3/envs/fitclone:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       1_gnu    conda-forge
atk                       2.36.0               ha770c72_4    conda-forge
atk-1.0                   2.36.0               h516909a_2    conda-forge
binutils_impl_linux-64    2.36.1               h193b22a_2    conda-forge
blas                      1.0                    openblas  
bzip2                     1.0.8                h7f98852_4    conda-forge
ca-certificates           2021.10.26           h06a4308_2    intel
cairo                     1.16.0            h18b612c_1001    conda-forge
cycler                    0.11.0             pyhd8ed1ab_0    conda-forge
cython                    0.29.21          py37he6710b0_0    anaconda
dbus                      1.13.6               he372182_0    conda-forge
expat                     2.4.7                h27087fc_0    conda-forge
fontconfig                2.13.96              h8e229c2_2    conda-forge
freetype                  2.10.4               h0708190_1    conda-forge
fribidi                   1.0.10               h36c2ea0_0    conda-forge
gcc                       11.2.0               h702ea55_8    conda-forge
gcc_impl_linux-64         11.2.0              h82a94d6_14    conda-forge
gdk-pixbuf                2.32.2                        1    bioconda
gettext                   0.19.8.1          hf34092f_1004    conda-forge
glib                      2.66.1               h680cd38_0    conda-forge
gobject-introspection     1.66.1           py37h619baee_1    conda-forge
graphite2                 1.3.13            h58526e2_1001    conda-forge
gstreamer                 1.14.5               h36ae1b5_2    conda-forge
gtk2                      2.24.32              h90f3771_0    conda-forge
harfbuzz                  2.4.0                h37c48d4_1    conda-forge
hdf5                      1.10.7               hb1b8bf9_0    anaconda
icu                       58.2              hf484d3e_1000    conda-forge
jbig                      2.1               h7f98852_2003    conda-forge
joblib                    1.0.1            py37h3f38642_0    intel
jpeg                      9e                   h7f98852_0    conda-forge
kernel-headers_linux-64   2.6.32              he073ed8_15    conda-forge
kiwisolver                1.4.0            py37h7cecad7_0    conda-forge
ld_impl_linux-64          2.36.1               hea4e1c9_2    conda-forge
lerc                      3.0                  h9c3ff4c_0    conda-forge
libblas                   3.8.0               17_openblas    conda-forge
libcblas                  3.8.0               17_openblas    conda-forge
libdeflate                1.10                 h7f98852_0    conda-forge
libffi                    3.2.1             he1b5a44_1007    conda-forge
libgcc                    7.2.0                h69d50b8_2    conda-forge
libgcc-devel_linux-64     11.2.0              h0952999_14    conda-forge
libgcc-ng                 11.2.0              h1d223b6_14    conda-forge
libgfortran-ng            7.5.0               h14aa051_20    conda-forge
libgfortran4              7.5.0               h14aa051_20    conda-forge
libgomp                   11.2.0              h1d223b6_14    conda-forge
libiconv                  1.16                 h516909a_0    conda-forge
liblapack                 3.8.0               17_openblas    conda-forge
liblapacke                3.8.0               17_openblas    conda-forge
libopenblas               0.3.10          pthreads_hb3c22a3_5    conda-forge
libpng                    1.6.37               h21135ba_2    conda-forge
libsanitizer              11.2.0              he4da1e4_14    conda-forge
libstdcxx-ng              11.2.0              he4da1e4_14    conda-forge
libtiff                   4.3.0                h542a066_3    conda-forge
libuuid                   2.32.1            h7f98852_1000    conda-forge
libwebp-base              1.2.2                h7f98852_1    conda-forge
libxcb                    1.13              h7f98852_1004    conda-forge
libxml2                   2.9.9                h13577e0_2    conda-forge
libzlib                   1.2.11            h36c2ea0_1013    conda-forge
llvm-openmp               8.0.1                hc9558a2_0    conda-forge
lz4-c                     1.9.3                h9c3ff4c_1    conda-forge
matplotlib                2.2.3            py37h8a2030e_1    conda-forge
matplotlib-base           2.2.3            py37h60b886d_1    conda-forge
ncurses                   6.3                  h9c3ff4c_0    conda-forge
numpy                     1.16.2           py37h8b7e671_1    conda-forge
numpy-base                1.17.0           py37h2f8d375_0    r
openmp                    8.0.1                         0    conda-forge
openssl                   1.0.2s                        0    intel
pandas                    0.24.2           py37hb3f55d8_1    conda-forge
pango                     1.42.4               h7062337_4    conda-forge
pcre                      8.45                 h9c3ff4c_0    conda-forge
pip                       22.0.4             pyhd8ed1ab_0    conda-forge
pixman                    0.38.0            h516909a_1003    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
pyparsing                 3.0.7              pyhd8ed1ab_0    conda-forge
pyqt                      5.6.0           py37h13b7fb3_1008    conda-forge
python                    3.7.1             hd21baee_1001    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python_abi                3.7                     2_cp37m    conda-forge
pytz                      2022.1             pyhd8ed1ab_0    conda-forge
pyyaml                    5.3.1            py37h5e8e339_2    conda-forge
qt                        5.6.2             hce4f676_1013    conda-forge
readline                  7.0               hf8c457e_1001    conda-forge
scikit-learn              0.22.2.post1     py37hcdab131_0    conda-forge
scipy                     1.1.0            py37he2b7bc3_2  
setuptools                59.8.0           py37h89c1867_0    conda-forge
sip                       4.18.1          py37hf484d3e_1000    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
sqlite                    3.28.0               h8b20d00_0    conda-forge
sysroot_linux-64          2.12                he073ed8_15    conda-forge
tk                        8.6.12               h27826a3_0    conda-forge
tornado                   6.1              py37h5e8e339_2    conda-forge
typing-extensions         4.1.1                hd8ed1ab_0    conda-forge
typing_extensions         4.1.1              pyha770c72_0    conda-forge
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
xarray                    0.11.0                py37_1000    conda-forge
xorg-kbproto              1.0.7             h7f98852_1002    conda-forge
xorg-libice               1.0.10               h7f98852_0    conda-forge
xorg-libsm                1.2.3             hd9c2040_1000    conda-forge
xorg-libx11               1.7.2                h7f98852_0    conda-forge
xorg-libxau               1.0.9                h7f98852_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xorg-libxext              1.3.4                h7f98852_1    conda-forge
xorg-libxrender           0.9.10            h7f98852_1003    conda-forge
xorg-libxt                1.2.1                h7f98852_2    conda-forge
xorg-renderproto          0.11.1            h7f98852_1002    conda-forge
xorg-xextproto            7.3.0             h7f98852_1002    conda-forge
xorg-xproto               7.0.31            h7f98852_1007    conda-forge
xz                        5.2.5                h516909a_1    conda-forge
yaml                      0.2.5                h7f98852_2    conda-forge
zlib                      1.2.11            h36c2ea0_1013    conda-forge
zstd                      1.5.2                ha95c52a_0    conda-forge
pcream commented 5 months ago

Hello, I've having the same problems with installation having gcc errors

Compiling wf_sample_parallel.pyx because it changed. [1/1] Cythonizing wf_sample_parallel.pyx /home/jcreamer/.conda/envs/fitness/lib/python3.7/site-packages/Cython/Compiler/Main.py:367: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /home/jcreamer/fitclone/fitclone/wf_sample_parallel.pyx tree = Parsing.p_module(s, pxd, full_module_name) In file included from wf_sample_parallel.c:23: /home/jcreamer/.conda/envs/fitness/include/python3.7m/Python.h:44:10: fatal error: crypt.h: No such file or directory 44 | #include | ^~~~~ compilation terminated. error: command '/home/jcreamer/.conda/envs/fitness/bin/gcc' failed with exit code 1

Here's my package list:

Name Version Build Channel _libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 2_gnu conda-forge binutils_impl_linux-64 2.40 hf600244_0 conda-forge blas 1.0 mkl bzip2 1.0.8 h5eee18b_5 c-ares 1.19.1 h5eee18b_0 anaconda ca-certificates 2024.2.2 hbcca054_0 intel certifi 2024.2.2 pyhd8ed1ab_0 intel cycler 0.11.0 pyhd3eb1b0_0 cython 0.29 py37he6710b0_0 dbus 1.13.18 hb2f20db_0 expat 2.5.0 h6a678d5_0 flit-core 3.6.0 pyhd3eb1b0_0 fontconfig 2.14.1 h52c9d5c_1 freetype 2.12.1 h4a9f257_0 gcc 13.2.0 hd6cf55c_3 conda-forge gcc_impl_linux-64 13.2.0 h338b0a0_5 conda-forge glib 2.78.4 h6a678d5_0 glib-tools 2.78.4 h6a678d5_0 gst-plugins-base 1.14.1 h6a678d5_1 gstreamer 1.14.1 h5eee18b_1 hdf5 1.12.1 h69dfa17_1 anaconda icu 58.2 he6710b0_3 intel-openmp 2021.4.0 h06a4308_3561 joblib 1.3.2 pyhd8ed1ab_0 intel jpeg 9e h5eee18b_1 kernel-headers_linux-64 2.6.32 he073ed8_17 conda-forge kiwisolver 1.4.4 py37h6a678d5_0 krb5 1.20.1 h568e23c_1 anaconda ld_impl_linux-64 2.40 h41732ed_0 conda-forge libcurl 7.88.1 h91b91d3_2 anaconda libedit 3.1.20221030 h5eee18b_0 anaconda libev 4.33 h7f8727e_1 anaconda libffi 3.4.4 h6a678d5_0 libgcc-devel_linux-64 13.2.0 ha9c7c90_105 conda-forge libgcc-ng 13.2.0 h807b86a_5 conda-forge libgfortran-ng 7.5.0 ha8ba4b0_17 libgfortran4 7.5.0 ha8ba4b0_17 libglib 2.78.4 hdc74915_0 libgomp 13.2.0 h807b86a_5 conda-forge libiconv 1.16 h7f8727e_2 libnghttp2 1.52.0 ha637b67_1 anaconda libpng 1.6.39 h5eee18b_0 libsanitizer 13.2.0 h7e041cc_5 conda-forge libssh2 1.10.0 h37d81fd_2 anaconda libstdcxx-ng 13.2.0 h7e041cc_5 conda-forge libuuid 1.41.5 h5eee18b_0 libxcb 1.15 h7f8727e_0 libxml2 2.9.14 h74e7548_0 matplotlib 2.2.3 py37hb69df0a_0 mkl 2020.2 256 mkl_fft 1.0.12 py37ha843d7b_0 mkl_random 1.1.1 py37h0573a6f_0 ncurses 6.4 h6a678d5_0 numpy 1.16.2 py37h7e9f1db_0 numpy-base 1.16.2 py37hde5b4d6_0 openssl 1.1.1w h7f8727e_0 anaconda pandas 0.24.2 py37he6710b0_0 pcre2 10.42 hebb0a14_0 pip 22.3.1 py37h06a4308_0 pyparsing 3.0.9 py37h06a4308_0 pyqt 5.9.2 py37h05f1152_2 python 3.7.16 h7a1cb2a_0 python-dateutil 2.8.2 pyhd3eb1b0_0 python_abi 3.7 1_cp37m conda-forge/label/cf202003 pytz 2022.7 py37h06a4308_0 pyyaml 5.3.1 py37h7b6447c_1 qt 5.9.7 h5867ecd_1 readline 8.2 h5eee18b_0 scikit-learn 0.22.2.post1 pypi_0 pypi scipy 1.1.0 py37h7c811a0_2 setuptools 65.6.3 py37h06a4308_0 sip 4.19.8 py37hf484d3e_0 six 1.16.0 pyhd3eb1b0_1 sqlite 3.41.2 h5eee18b_0 sysroot_linux-64 2.12 he073ed8_17 conda-forge threadpoolctl 3.1.0 pyh8a188c0_0 intel tk 8.6.12 h1ccaba5_0 tornado 6.2 py37h5eee18b_0 typing_extensions 4.4.0 py37h06a4308_0 wheel 0.38.4 py37h06a4308_0 xarray 0.11.0 py37_0 xz 5.4.6 h5eee18b_0 yaml 0.2.5 h7b6447c_0 zlib 1.2.13 h5eee18b_0

@vleblanc were you able to get the package running? I'm worried that it might be in a non-functional state since the maintainer hasn't commented on any issues for a while now.

vleblanc commented 5 months ago

No, I wasn't able to get it running, sorry

sohrabsa commented 5 months ago

Hi! What OS are you trying to install fitClone on?

pcream commented 5 months ago

Hello! I'm running it on Ubuntu.

On Wed, Apr 3, 2024, 6:42 PM Sohrab @.***> wrote:

Hi! What OS are you trying to install fitClone on?

— Reply to this email directly, view it on GitHub https://github.com/UBC-Stat-ML/fitclone/issues/12#issuecomment-2035959949, or unsubscribe https://github.com/notifications/unsubscribe-auth/AY4TCXTUXH7YM4PIM75A7MTY3SVZLAVCNFSM5RSRZMEKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMBTGU4TKOJZGQ4Q . You are receiving this because you commented.Message ID: @.***>

sohrabsa commented 5 months ago

@vleblanc I confirm that I can recreate the error on Big Sur. Running ./cyconvert_bigsur.sh fixes that.

sohrabsa commented 5 months ago

Hello! I'm running it on Ubuntu. … On Wed, Apr 3, 2024, 6:42 PM Sohrab @.> wrote: Hi! What OS are you trying to install fitClone on? — Reply to this email directly, view it on GitHub <#12 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AY4TCXTUXH7YM4PIM75A7MTY3SVZLAVCNFSM5RSRZMEKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMBTGU4TKOJZGQ4Q . You are receiving this because you commented.Message ID: @.>

Okay, this seems like a problem with glibc installation (or lack there of). Sadly I don't have access to ubuntu right now. Searching around you may need the libc6-dev package, something like sudo apt-get install libc6-dev. Then attempt a clean reinstall of fitClone. Let me know how it goes.

pcream commented 5 months ago

Ok, I checked and it seems like libc6-dev is already installed:

Reading package lists... Done Building dependency tree Reading state information... Done libc6-dev is already the newest version (2.31-0ubuntu9.14). libc6-dev set to manually installed. The following packages were automatically installed and are no longer required: libfwupdplugin1 libncurses5-dev libxmlb1 Use 'sudo apt autoremove' to remove them. 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

I removed the old conda env and I'm trying a clean install of fitClone. I also noticed during installation that it is having issues getting this specific version of scikit_learn and gcc while installed:

PackagesNotFoundError: The following packages are not available from current channels:

  • scikit_learn==0.22.2.post1

Current channels:

PackagesNotFoundError: The following packages are not available from current channels:

  • gcc

Current channels:

Installation ends with the same error messages as above. The gcc error is confusing because it is definitely already installed and callable.

Thanks for your help!!

pcream commented 5 months ago

I think I might have figured out the issue, libcrypt is now unbundled from glibc discussed here and is now it's own package libxcrypt. So I installed libxcrypt and then copied the crypt.h file to the envs/fitness/include/python3.7m/ directory that it's expecting it to be in as suggested by this post. I then was able to get ./fitclone.py running after running pip install tornado to get around the ModuleNotFoundError: No module named 'tornado'. It then produced an output folder with infer_theta.tsv.gz and the other expected files, so I believe it is working now, but I need to install the correct R packages to get the summary plots generated.

Edit: Thanks for you help!