jeffspence
commented
1 month ago Hmmm -- that's a tough one to debug. Do you get errors when you run the example.sh script in the example directory?
Open TakashiKoyama opened 1 month ago
jeffspence
commented
1 month ago Hmmm -- that's a tough one to debug. Do you get errors when you run the example.sh script in the example directory?
jeffspence
commented
1 month ago Could also be a caching issue with numba --- which version of numba do you have (e.g., by running pip freeze | grep numba)?
TakashiKoyama
commented
1 month ago @jeffspence
Thank you for your reply.
example.sh does not work properly. The end of stdout say HDF version is wrong.
Am I correct?
INFO:root:Finished likelihoods in 0.723522 seconds for rho=0.000000
INFO:root:Cleaning up results...
INFO:root:Computed lookup table in 152.569932 seconds
INFO:root: ...complete
INFO:root:Downsampling
INFO:root:Downsampling... Currently at n = 25
INFO:root:Downsampling... Currently at n = 24
INFO:root:Downsampling... Currently at n = 23
INFO:root:Downsampling... Currently at n = 22
INFO:root:Downsampling... Currently at n = 21
INFO:root: ...complete
Warning! ***HDF5 library version mismatched error***
The HDF5 header files used to compile this application do not match
the version used by the HDF5 library to which this application is linked.
Data corruption or segmentation faults may occur if the application continues.
This can happen when an application was compiled by one version of HDF5 but
linked with a different version of static or shared HDF5 library.
You should recompile the application or check your shared library related
settings such as 'LD_LIBRARY_PATH'.
You can, at your own risk, disable this warning by setting the environment
variable 'HDF5_DISABLE_VERSION_CHECK' to a value of '1'.
Setting it to 2 or higher will suppress the warning messages totally.
Headers are 1.12.2, library is 1.14.2
SUMMARY OF THE HDF5 CONFIGURATION
=================================
General Information:
-------------------
HDF5 Version: 1.14.2
Configured on: Tue Mar 26 22:23:22 JST 2024
Configured by: root@XW-E5S.suijitsu.utokyo.jp
Host system: x86_64-pc-linux-gnu
Uname information: Linux XW-E5S.suijitsu.utokyo.jp 3.10.0-514.10.2.el7.x86_64 #1 SMP Fri Mar 3 00:04:05 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
Byte sex: little-endian
Installation point: /usr/local/hdf5
Compiling Options:
------------------
Build Mode: production
Debugging Symbols: no
Asserts: no
Profiling: no
Optimization Level: high
Linking Options:
----------------
Libraries: static, shared
Statically Linked Executables:
LDFLAGS:
H5_LDFLAGS:
AM_LDFLAGS:
Extra libraries: -lpthread -lz -ldl -lm
Archiver: ar
AR_FLAGS: cr
Ranlib: ranlib
Languages:
----------
C: yes
C Compiler: /opt/rh/devtoolset-8/root/usr/bin/gcc ( gcc (GCC) 8.3.1 20190311 )
CPPFLAGS:
H5_CPPFLAGS: -D_GNU_SOURCE -D_POSIX_C_SOURCE=200809L -DNDEBUG -UH5_DEBUG_API -I/root/Downloads/hdfsrc/src/H5FDsubfiling
AM_CPPFLAGS:
C Flags:
H5 C Flags: -std=c99 -Wall -Wcast-qual -Wconversion -Wextra -Wfloat-equal -Wformat=2 -Winit-self -Winvalid-pch -Wmissing-include-dirs -Wshadow -Wundef -Wwrite-strings -pedantic -Wno-c++-compat -Wlarger-than=2560 -Wlogical-op -Wframe-larger-than=16384 -Wpacked-bitfield-compat -Wsync-nand -Wno-unsuffixed-float-constants -Wdouble-promotion -Wtrampolines -Wstack-usage=8192 -Wmaybe-uninitialized -Wdate-time -Warray-bounds=2 -Wc99-c11-compat -Wduplicated-cond -Whsa -Wnormalized -Wnull-dereference -Wunused-const-variable -Walloca -Walloc-zero -Wduplicated-branches -Wformat-overflow=2 -Wformat-truncation=1 -Wattribute-alias -Wcast-align=strict -Wshift-overflow=2 -fstdarg-opt -s -Wbad-function-cast -Wcast-align -Wformat -Wimplicit-function-declaration -Wint-to-pointer-cast -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wold-style-definition -Wpacked -Wpointer-sign -Wpointer-to-int-cast -Wredundant-decls -Wstrict-prototypes -Wswitch -Wunused-but-set-variable -Wunused-variable -Wunused-function -Wunused-parameter -Wincompatible-pointer-types -Wint-conversion -Wshadow -Wrestrict -Wcast-function-type -Wmaybe-uninitialized -Wno-aggregate-return -Wno-inline -Wno-missing-format-attribute -Wno-missing-noreturn -Wno-overlength-strings -Wno-jump-misses-init -Wstrict-overflow=2 -Wno-suggest-attribute=const -Wno-suggest-attribute=noreturn -Wno-suggest-attribute=pure -Wno-suggest-attribute=format -Wno-suggest-attribute=cold -Wno-suggest-attribute=malloc -O3
AM C Flags:
Shared C Library: yes
Static C Library: yes
Fortran: no
C++: no
Java: no
Features:
---------
Parallel HDF5: no
Parallel Filtered Dataset Writes: no
Large Parallel I/O: no
High-level library: yes
Dimension scales w/ new references: no
Build HDF5 Tests: yes
Build HDF5 Tools: yes
Build GIF Tools: no
Threadsafety: yes
Default API mapping: v114
With deprecated public symbols: yes
I/O filters (external): deflate(zlib)
Map (H5M) API: no
Direct VFD: no
Mirror VFD: no
Subfiling VFD: no
(Read-Only) S3 VFD: no
(Read-Only) HDFS VFD: no
Packages w/ extra debug output: none
API tracing: no
Using memory checker: no
Function stack tracing: no
Use file locking: best-effort
Strict file format checks: no
Optimization instrumentation: no
Bye...
example.sh: line 30: 12929 Aborted (core dumped) pyrho make_table -n 20 -N 25 --mu 1.25e-8 --logfile . --outfile ACB_n_20_N_40_lookuptable.hdf --approx --smcpp_file ACB_pop_sizes.csv --decimate_rel_tol 0.1
Traceback (most recent call last):
File "/usr/local/src/pyenv/versions/3.8.11/bin/pyrho", line 8, in <module>
sys.exit(main())
File "/usr/local/src/pyenv/versions/3.8.11/lib/python3.8/site-packages/pyrho/frontend.py", line 67, in main
func(args)
File "/usr/local/src/pyenv/versions/3.8.11/lib/python3.8/site-packages/pyrho/hyperparameter_optimizer.py", line 194, in _main
table = read_hdf(args.tablefile, 'ldtable')
File "/usr/local/src/pyenv/versions/3.8.11/lib/python3.8/site-packages/pandas/io/pytables.py", line 418, in read_hdf
raise FileNotFoundError(f"File {path_or_buf} does not exist")
FileNotFoundError: File ACB_n_20_N_40_lookuptable.hdf does not exist
INFO:root:Reading data from ACB_chr_1_subset.vcf.gz
INFO:root: Done!
INFO:root:Loading lookup table...
Traceback (most recent call last):
File "/usr/local/src/pyenv/versions/3.8.11/bin/pyrho", line 8, in <module>
sys.exit(main())
File "/usr/local/src/pyenv/versions/3.8.11/lib/python3.8/site-packages/pyrho/frontend.py", line 67, in main
func(args)
File "/usr/local/src/pyenv/versions/3.8.11/lib/python3.8/site-packages/pyrho/optimizer.py", line 350, in _main
table = read_hdf(args.tablefile, 'ldtable')
File "/usr/local/src/pyenv/versions/3.8.11/lib/python3.8/site-packages/pandas/io/pytables.py", line 418, in read_hdf
raise FileNotFoundError(f"File {path_or_buf} does not exist")
FileNotFoundError: File ACB_n_20_N_40_lookuptable.hdf does not exist
Traceback (most recent call last):
File "/usr/local/src/pyenv/versions/3.8.11/bin/pyrho", line 8, in <module>
sys.exit(main())
File "/usr/local/src/pyenv/versions/3.8.11/lib/python3.8/site-packages/pyrho/frontend.py", line 67, in main
func(args)
File "/usr/local/src/pyenv/versions/3.8.11/lib/python3.8/site-packages/pyrho/compute_r2.py", line 97, in _main
table = DataFrame(read_hdf(args.tablefile, 'ldtable'))
File "/usr/local/src/pyenv/versions/3.8.11/lib/python3.8/site-packages/pandas/io/pytables.py", line 418, in read_hdf
raise FileNotFoundError(f"File {path_or_buf} does not exist")
FileNotFoundError: File ACB_n_20_N_40_lookuptable.hdf does not exist@jeffspence
The version of number is numba==0.58.1.
jeffspence
commented
1 month ago Hi @TakashiKoyama -- yes, that looks like some sort of mismatch in hdf5 versioning (and not a pyrho problem, per se). This SO post suggests that maybe resinstalling h5py could help: https://stackoverflow.com/questions/45411700/warning-hdf5-library-version-mismatched-error-python-pandas-windows. It might also be necessary to update or downgrade the version of hfd5 on your system.
Hello, I successfully installed pyrho using conda as described in README but got error when running tests.py. The error messages are as below. Could anyone help me solving this problem? Thanks.