search

luojunwei / BreakNet

BreakNet: detecting deletions using long reads and a deep learning apporch
GNU General Public License v3.0
4 stars 2 forks source link

Error while executing breaknet.py: ImportError: cannot import name 'train_fn' from 'model' #4

Open ngs1810 opened 1 year ago

ngs1810 commented 1 year ago

I am trying to execute the breaknet.py, however, faced this issue. I did load all modules required as dependencies.

The error message upon execution:- ImportError: cannot import name 'train_fn' from 'model'

Is there any advice for this? Thank you in advance

micahvista commented 1 year ago

Try this. https://github.com/micahvista/BreakNet

ngs1810 commented 1 year ago

Thank you. I still face some problems. Apologies if these are naive ones.

This is my command in the script:-

python3.7 "$BreakNet/code/breaknet.py" call_mode "$OUTDIR" "$BreakNet/trained_weight/weight.data-00000-of-00001" "$ONT/20230517_gDNA_ONT.GRCh38.merged.bam"

And the log output and/or error messages are like this:-

2023-08-18 18:57:11.667513: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0 2023-08-18 18:58:02.008089: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set 2023-08-18 18:58:02.037991: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /apps/software/Python/3.7.0-foss-2016b/lib/python3.7/site-packages/numpy-1.14.5-py3.7-linux-x86_64.egg/numpy/core/lib:/apps/software/Python/3.7.0-foss-2016b/lib:/apps/software/libffi/3.2.1-foss-2016b/lib64:/apps/software/libffi/3.2.1-foss-2016b/lib:/apps/software/GMP/6.1.1-foss-2016b/lib:/apps/software/XZ/5.2.2-foss-2016b/lib:/apps/software/SQLite/3.13.0-foss-2016b/lib:/apps/software/Tcl/8.6.5-foss-2016b/lib:/apps/software/libreadline/6.3-foss-2016b/lib:/apps/software/ncurses/6.0-foss-2016b/lib:/apps/software/zlib/1.2.8-foss-2016b/lib:/apps/software/bzip2/1.0.6-foss-2016b/lib:/apps/software/ScaLAPACK/2.0.2-gompi-2016b-OpenBLAS-0.2.18-LAPACK-3.6.1/lib:/apps/software/FFTW/3.3.4-gompi-2016b/lib:/apps/software/OpenBLAS/0.2.18-GCC-5.4.0-2.26-LAPACK-3.6.1/lib:/apps/software/OpenMPI/1.10.3-GCC-5.4.0-2.26/lib:/apps/software/hwloc/1.11.3-GCC-5.4.0-2.26/lib:/apps/software/numactl/2.0.11-GCC-5.4.0-2.26/lib:/apps/software/binutils/2.26-GCCcore-5.4.0/lib:/apps/software/GCCcore/5.4.0/lib/gcc/x86_64-unknown-linux-gnu/5.4.0:/apps/software/GCCcore/5.4.0/lib64:/apps/software/GCCcore/5.4.0/lib 2023-08-18 18:58:02.038564: W tensorflow/stream_executor/cuda/cuda_driver.cc:326] failed call to cuInit: UNKNOWN ERROR (303) 2023-08-18 18:58:02.038887: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (p1-cpu-43): /proc/driver/nvidia/version does not exist 2023-08-18 18:58:05.888013: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX512F To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2023-08-18 18:58:05.893043: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set 2023-08-18 18:58:09.036112: W tensorflow/core/util/tensor_slice_reader.cc:95] Could not open /hpcfs/users/a1742674/Chr22q11.2/BreakNet/BreakNet-main/trained_weight/weight.data-00000-of-00001: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator? Model: "model"

Layer (type) Output Shape Param #

=================================================================

input_1 (InputLayer) [(None, None, 200, 18, 1) 0

time_distributed (TimeDistri (None, None, 160) 123812

bidirectional (Bidirectional (None, None, 128) 115200

bidirectional_1 (Bidirection (None, None, 128) 98816

dense (Dense) (None, None, 64) 8256

dropout (Dropout) (None, None, 64) 0

dense_1 (Dense) (None, None, 64) 4160

dropout_1 (Dropout) (None, None, 64) 0

dense_2 (Dense) (None, None, 1) 65

================================================================= Total params: 350,309 Trainable params: 350,309 Non-trainable params: 0

testing bamfile path /HOME/20230517_gDNA_ONT.GRCh38.merged.bam weight path /hpc/USERS/a1d/BreakNet-main/trained_weight/weight.data-00000-of-00001 data file path /hpc/USERS/a1d/BreakNet-main/call_folder All chromosomes within bamfile will be used Model: "model_1"

Layer (type) Output Shape Param #

================================================================= input_2 (InputLayer) [(None, None, 200, 18, 1) 0

time_distributed_1 (TimeDist (None, None, 160) 123812

bidirectional_2 (Bidirection (None, None, 128) 115200

bidirectional_3 (Bidirection (None, None, 128) 98816

dense_3 (Dense) (None, None, 64) 8256

dropout_2 (Dropout) (None, None, 64) 0

dense_4 (Dense) (None, None, 64) 4160

dropout_3 (Dropout) (None, None, 64) 0

dense_5 (Dense) (None, None, 1) 65

================================================================= Total params: 350,309 Trainable params: 350,309 Non-trainable params: 0

Traceback (most recent call last): File "/hpc/USERS/a1d/BreakNet-main/code/breaknet.py", line 55, in predict_fn(datapath = datapath, weightpath = weightpath, bamfilepath = bamfilepath, includecontig=includecontig ) File "/hpc/USERS/a1d/BreakNet-main/code/model.py", line 1222, in predict_fn model.load_weights(weightpath) File "/hpc/USERS/a1d/BreakNet-main/venv/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py", line 2227, in load_weights with h5py.File(filepath, 'r') as f: File "/hpc/USERS/a1d/BreakNet-main/venv/lib/python3.7/site-packages/h5py/_hl/files.py", line 408, in init swmr=swmr) File "/hpc/USERS/a1d/BreakNet-main/venv/lib/python3.7/site-packages/h5py/_hl/files.py", line 173, in make_fid fid = h5f.open(name, flags, fapl=fapl) File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper File "h5py/h5f.pyx", line 88, in h5py.h5f.open OSError: Unable to open file (file signature not found)

micahvista commented 1 year ago

Sorry for the late response, try this command python3.7 "$BreakNet/code/breaknet.py" call_mode "$OUTDIR" "$BreakNet/trained_weight/weight" "$ONT/20230517_gDNA_ONT.GRCh38.merged.bam"

Sent from my iPhone

------------------ Original ------------------ From: ngs1810 @.> Date: Fri, Aug 18, 2023 5:37 PM To: luojunwei/BreakNet @.> Cc: DingHongyu @.>, Comment @.> Subject: Re: [luojunwei/BreakNet] Error while executing breaknet.py:ImportError: cannot import name 'train_fn' from 'model' (Issue #4)

Thank you. I still face some problems. Apologies if these are naive ones.

This is my command in the script:-

python3.7 "$BreakNet/code/breaknet.py" call_mode "$OUTDIR" "$BreakNet/trained_weight/weight.data-00000-of-00001" "$ONT/20230517_gDNA_ONT.GRCh38.merged.bam"

And the log output and/or error messages are like this:-

2023-08-18 18:57:11.667513: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0 2023-08-18 18:58:02.008089: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set 2023-08-18 18:58:02.037991: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /apps/software/Python/3.7.0-foss-2016b/lib/python3.7/site-packages/numpy-1.14.5-py3.7-linux-x86_64.egg/numpy/core/lib:/apps/software/Python/3.7.0-foss-2016b/lib:/apps/software/libffi/3.2.1-foss-2016b/lib64:/apps/software/libffi/3.2.1-foss-2016b/lib:/apps/software/GMP/6.1.1-foss-2016b/lib:/apps/software/XZ/5.2.2-foss-2016b/lib:/apps/software/SQLite/3.13.0-foss-2016b/lib:/apps/software/Tcl/8.6.5-foss-2016b/lib:/apps/software/libreadline/6.3-foss-2016b/lib:/apps/software/ncurses/6.0-foss-2016b/lib:/apps/software/zlib/1.2.8-foss-2016b/lib:/apps/software/bzip2/1.0.6-foss-2016b/lib:/apps/software/ScaLAPACK/2.0.2-gompi-2016b-OpenBLAS-0.2.18-LAPACK-3.6.1/lib:/apps/software/FFTW/3.3.4-gompi-2016b/lib:/apps/software/OpenBLAS/0.2.18-GCC-5.4.0-2.26-LAPACK-3.6.1/lib:/apps/software/OpenMPI/1.10.3-GCC-5.4.0-2.26/lib:/apps/software/hwloc/1.11.3-GCC-5.4.0-2.26/lib:/apps/software/numactl/2.0.11-GCC-5.4.0-2.26/lib:/apps/software/binutils/2.26-GCCcore-5.4.0/lib:/apps/software/GCCcore/5.4.0/lib/gcc/x86_64-unknown-linux-gnu/5.4.0:/apps/software/GCCcore/5.4.0/lib64:/apps/software/GCCcore/5.4.0/lib 2023-08-18 18:58:02.038564: W tensorflow/stream_executor/cuda/cuda_driver.cc:326] failed call to cuInit: UNKNOWN ERROR (303) 2023-08-18 18:58:02.038887: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (p1-cpu-43): /proc/driver/nvidia/version does not exist 2023-08-18 18:58:05.888013: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX512F To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2023-08-18 18:58:05.893043: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set 2023-08-18 18:58:09.036112: W tensorflow/core/util/tensor_slice_reader.cc:95] Could not open /hpcfs/users/a1742674/Chr22q11.2/BreakNet/BreakNet-main/trained_weight/weight.data-00000-of-00001: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator? Model: "model"

Layer (type) Output Shape Param #

input_1 (InputLayer) [(None, None, 200, 18, 1) 0

time_distributed (TimeDistri (None, None, 160) 123812

bidirectional (Bidirectional (None, None, 128) 115200

bidirectional_1 (Bidirection (None, None, 128) 98816

dense (Dense) (None, None, 64) 8256

dropout (Dropout) (None, None, 64) 0

dense_1 (Dense) (None, None, 64) 4160

dropout_1 (Dropout) (None, None, 64) 0

dense_2 (Dense) (None, None, 1) 65

Total params: 350,309 Trainable params: 350,309 Non-trainable params: 0

testing bamfile path /HOME/20230517_gDNA_ONT.GRCh38.merged.bam weight path /hpc/USERS/a1d/BreakNet-main/trained_weight/weight.data-00000-of-00001 data file path /hpc/USERS/a1d/BreakNet-main/call_folder All chromosomes within bamfile will be used Model: "model_1"

Layer (type) Output Shape Param #

input_2 (InputLayer) [(None, None, 200, 18, 1) 0

time_distributed_1 (TimeDist (None, None, 160) 123812

bidirectional_2 (Bidirection (None, None, 128) 115200

bidirectional_3 (Bidirection (None, None, 128) 98816

dense_3 (Dense) (None, None, 64) 8256

dropout_2 (Dropout) (None, None, 64) 0

dense_4 (Dense) (None, None, 64) 4160

dropout_3 (Dropout) (None, None, 64) 0

dense_5 (Dense) (None, None, 1) 65

Total params: 350,309 Trainable params: 350,309 Non-trainable params: 0

Traceback (most recent call last): File "/hpc/USERS/a1d/BreakNet-main/code/breaknet.py", line 55, in predict_fn(datapath = datapath, weightpath = weightpath, bamfilepath = bamfilepath, includecontig=includecontig ) File "/hpc/USERS/a1d/BreakNet-main/code/model.py", line 1222, in predict_fn model.load_weights(weightpath) File "/hpc/USERS/a1d/BreakNet-main/venv/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py", line 2227, in load_weights with h5py.File(filepath, 'r') as f: File "/hpc/USERS/a1d/BreakNet-main/venv/lib/python3.7/site-packages/h5py/_hl/files.py", line 408, in init swmr=swmr) File "/hpc/USERS/a1d/BreakNet-main/venv/lib/python3.7/site-packages/h5py/_hl/files.py", line 173, in make_fid fid = h5f.open(name, flags, fapl=fapl) File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper File "h5py/h5f.pyx", line 88, in h5py.h5f.open OSError: Unable to open file (file signature not found)

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

micahvista commented 6 months ago

Please use this version https://github.com/luojunwei/BreakNet/tree/af7d77d85cbb758928268851323fa07eb7efc1de to label data and train the model.