error

[rojinsafavi]

Hello, I keep getting this error, I would appriciate if I can get some help.

(chiron-env) -bash-4.2$ python chiron/chiron_rcnn_train.py I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcublas.so.8.0 locally I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcudnn.so.5 locally I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcufft.so.8.0 locally I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcuda.so.1 locally I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcurand.so.8.0 locally (400, 1, 300, 1) [400, 300, 256] Traceback (most recent call last): File "chiron/chiron_rcnn_train.py", line 131, in run(flags) File "chiron/chiron_rcnn_train.py", line 115, in run train() File "chiron/chiron_rcnn_train.py", line 70, in train opt = train_step(ctc_loss,global_step = global_step) TypeError: train_step() takes at least 2 arguments (2 given) (chiron-env) -bash-4.2$

[haotianteng]

The Github repository is currently under development, please download the stable release version(v0.3). From Pypi website: https://pypi.python.org/pypi/chiron

[haotianteng]

And also for your information, a major update will come recently, will include a new trained model which has higher accuracy on Human dataset and a RNA model which can be used to basecall RNA.

[rojinsafavi]

Thanks Hao, the training worked ( I got some output). Would you please guid me through how to do evaluation? to measure the accuracy of the basecalling? I tired chiron-eval.py, but the output folders are empty

(chiron-env) bash-4.2$ python chiron_eval.py -i /RNA-eval/ -o /chiron-eval-output/ -m /GVM_model/res50/model/model/DNA_default/ -t 1
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcublas.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcudnn.so.5 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcufft.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcurand.so.8.0 locally
(1100, 1, 300, 1)
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE3 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
I tensorflow/core/common_runtime/gpu/gpu_device.cc:885] Found device 0 with properties: 
name: Tesla K20c
major: 3 minor: 5 memoryClockRate (GHz) 0.7055
pciBusID 0000:03:00.0
Total memory: 4.63GiB
Free memory: 4.56GiB
W tensorflow/stream_executor/cuda/cuda_driver.cc:590] creating context when one is currently active; existing: 0x4f9a540
I tensorflow/core/common_runtime/gpu/gpu_device.cc:885] Found device 1 with properties: 
name: Tesla K20c
major: 3 minor: 5 memoryClockRate (GHz) 0.7055
pciBusID 0000:43:00.0
Total memory: 4.63GiB
Free memory: 4.56GiB
I tensorflow/core/common_runtime/gpu/gpu_device.cc:777] Peer access not supported between device ordinals 0 and 1
I tensorflow/core/common_runtime/gpu/gpu_device.cc:777] Peer access not supported between device ordinals 1 and 0
I tensorflow/core/common_runtime/gpu/gpu_device.cc:906] DMA: 0 1 
I tensorflow/core/common_runtime/gpu/gpu_device.cc:916] 0:   Y N 
I tensorflow/core/common_runtime/gpu/gpu_device.cc:916] 1:   N Y 
I tensorflow/core/common_runtime/gpu/gpu_device.cc:975] Creating TensorFlow device (/gpu:0) -> (device: 0, name: Tesla K20c, pci bus id: 0000:03:00.0)
I tensorflow/core/common_runtime/gpu/gpu_device.cc:975] Creating TensorFlow device (/gpu:1) -> (device: 1, name: Tesla K20c, pci bus id: 0000:43:00.0)
/chiron-eval-output/
Real time:4.157 Systime:1.429 Usertime:1.378
(chiron-env) bash-4.2$ 

Thank you in advance.

[haotianteng]

Hi Rojin, The easiest way would be telling the Chiron the model you want to use for evaluation by the --model flag.

Thanks Teng

[rojinsafavi]

Dear Tang, I used chiron-eval.py and gave it the model folder as follow:

(chiron-env) bash-4.2$ python chiron_eval.py -i /RNA-eval/ -o /chiron-eval-output/ -m /GVM_model/res50/model/model/DNA_default/ -t 1
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcublas.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcudnn.so.5 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcufft.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcurand.so.8.0 locally
(1100, 1, 300, 1)
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE3 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
I tensorflow/core/common_runtime/gpu/gpu_device.cc:885] Found device 0 with properties: 
name: Tesla K20c
major: 3 minor: 5 memoryClockRate (GHz) 0.7055
pciBusID 0000:03:00.0
Total memory: 4.63GiB
Free memory: 4.56GiB
W tensorflow/stream_executor/cuda/cuda_driver.cc:590] creating context when one is currently active; existing: 0x4f9a540
I tensorflow/core/common_runtime/gpu/gpu_device.cc:885] Found device 1 with properties: 
name: Tesla K20c
major: 3 minor: 5 memoryClockRate (GHz) 0.7055
pciBusID 0000:43:00.0
Total memory: 4.63GiB
Free memory: 4.56GiB
I tensorflow/core/common_runtime/gpu/gpu_device.cc:777] Peer access not supported between device ordinals 0 and 1
I tensorflow/core/common_runtime/gpu/gpu_device.cc:777] Peer access not supported between device ordinals 1 and 0
I tensorflow/core/common_runtime/gpu/gpu_device.cc:906] DMA: 0 1 
I tensorflow/core/common_runtime/gpu/gpu_device.cc:916] 0:   Y N 
I tensorflow/core/common_runtime/gpu/gpu_device.cc:916] 1:   N Y 
I tensorflow/core/common_runtime/gpu/gpu_device.cc:975] Creating TensorFlow device (/gpu:0) -> (device: 0, name: Tesla K20c, pci bus id: 0000:03:00.0)
I tensorflow/core/common_runtime/gpu/gpu_device.cc:975] Creating TensorFlow device (/gpu:1) -> (device: 1, name: Tesla K20c, pci bus id: 0000:43:00.0)
/chiron-eval-output/
Real time:4.157 Systime:1.429 Usertime:1.378
(chiron-env) bash-4.2$ 

the output has 3 folders, but 2 of them are empty.

Am I missing something here? Kind regards, Rojin

[haotianteng]

Are you sure the input path is /RNA-eval? Should it be ./RNA-eval?

[rojinsafavi]

This is the full command that I'm running: the input path is the path to fast5 files for evaluation. The output has three folders, except for the meta folder the rest are empty

(chiron-env) -bash-4.2$ python chiron_eval.py -i ../../RNA-eval/ -o ../../chiron-eval-output/ -m ../../GVM_model/res50/model/model/DNA_default/ -t 1

I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcublas.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcudnn.so.5 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcufft.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcurand.so.8.0 locally
(1100, 1, 300, 1)
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE3 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
I tensorflow/core/common_runtime/gpu/gpu_device.cc:885] Found device 0 with properties: 
name: Tesla K20c
major: 3 minor: 5 memoryClockRate (GHz) 0.7055
pciBusID 0000:03:00.0
Total memory: 4.63GiB
Free memory: 4.56GiB
W tensorflow/stream_executor/cuda/cuda_driver.cc:590] creating context when one is currently active; existing: 0x652c280
I tensorflow/core/common_runtime/gpu/gpu_device.cc:885] Found device 1 with properties: 
name: Tesla K20c
major: 3 minor: 5 memoryClockRate (GHz) 0.7055
pciBusID 0000:43:00.0
Total memory: 4.63GiB
Free memory: 4.56GiB
I tensorflow/core/common_runtime/gpu/gpu_device.cc:777] Peer access not supported between device ordinals 0 and 1
I tensorflow/core/common_runtime/gpu/gpu_device.cc:777] Peer access not supported between device ordinals 1 and 0
I tensorflow/core/common_runtime/gpu/gpu_device.cc:906] DMA: 0 1 
I tensorflow/core/common_runtime/gpu/gpu_device.cc:916] 0:   Y N 
I tensorflow/core/common_runtime/gpu/gpu_device.cc:916] 1:   N Y 
I tensorflow/core/common_runtime/gpu/gpu_device.cc:975] Creating TensorFlow device (/gpu:0) -> (device: 0, name: Tesla K20c, pci bus id: 0000:03:00.0)
I tensorflow/core/common_runtime/gpu/gpu_device.cc:975] Creating TensorFlow device (/gpu:1) -> (device: 1, name: Tesla K20c, pci bus id: 0000:43:00.0)
./../../chiron-eval-output/
Real time:4.630 Systime:1.682 Usertime:1.377
(chiron-env) -bash-4.2$ 

[haotianteng]

An empty raw folder indicates that the signal is not extracted, which means the input path is not correct. Use absolute path.

[rojinsafavi]

I'm not seeing in raw folder though, it is only meta, result, and segments folders within the chiron-eval-output output folder.

(chiron-env) -bash-4.2$ python chiron_eval.py -i /projects/nanopore-working/rojin/RNA-eval/ -o /projects/nanopore-working/rojin/chiron-eval-output/ -m /projects/nanopore-working/rojin/GVM_model/res50/model/model/DNA_default/ -t 1

Still getting the same error. Is the input folder supposed to be a folder contacting fast5 files? or should extract raw signal from those fast5 files and give that as the input?

Thanks Teng

[haotianteng]

Yes, the raw signal should be extracted first if you use chiron_eval.py directly. Use chiron/utils/raw.py in advance to create the raw folder, it should contain .signal files, and then use chiron_eval.py on this raw folder.

If you use entry.py or install from pip(chiron call -i fast5 folder), then just call on the fast5 folder, but it still call the raw.py to extract the .signal file.

[haotianteng]

In your case run this command: python chiron/utils/extract_sig_ref.py --input=/projects/nanopore-working/rojin/RNA-eval/ --output=/projects/nanopore-working/rojin/RNA-eval/

And then run chiron_eval.py on the raw folder. python chiron/chiron_eval.py --model=[Path to Your model] --input=/projects/nanopore-working/rojin/RNA-eval/raw --output=/projects/nanopore-working/rojin/RNA-eval/output

However, I don't expect Chiron version 0.3 will work directly on RNA basecalling, as the pore model is different of RNA and DNA, and also the RNA signal is reversed in the fast5 file while DNA signal is not.