search

haotianteng / Chiron

A basecaller for Oxford Nanopore Technologies' sequencers
Other
122 stars 53 forks source link

unable to run 'chiron/entry.py call' #30

Closed AEDWIP closed 6 years ago

AEDWIP commented 6 years ago

I am getting a bunch of errors when I try to call using the sample data. I wonder if the pretrained model is missing?

Kind regards

Andy

(Chiron_env) $ gb
* master 548c493 [origin/master] Docstring in chiron_train module.

(Chiron_env) $ pwd
/Users/ad/workSpace/UCSC/nanoPore/Chiron

(Chiron_env) $ python chiron/entry.py call -i chiron/example_data/ -o ../Chiron.out/ 2>&1 | tee entry.py.call.error.out.txt

generates a lot of messages like

W tensorflow/core/framework/op_kernel.cc:993] Not found: Key block1dilate_layer1/dilate_branch/filter_branch/filter/weights not found in checkpoint
W tensorflow/core/framework/op_kernel.cc:993] Not found: Key block1dilate_layer1/dilate_branch/filter_branch/filter_bn/filter_bn_offset not found in checkpoint
W tensorflow/core/framework/op_kernel.cc:993] Not found: Key block1dilate_layer1/dilate_branch/filter_branch/filter_bn/filter_bn_scale not found in checkpoint

eventually I get the following stack trace

Caused by op u'save/RestoreV2_12', defined at:
  File "chiron/entry.py", line 86, in <module>
    main()
  File "chiron/entry.py", line 81, in main
    args.func(args)
  File "chiron/entry.py", line 23, in evaluation
    chiron_eval.run(args)
  File "/Users/ad/workSpace/UCSC/nanoPore/Chiron/chiron/chiron_eval.py", line 228, in run
    time_dict = unix_time(evaluation)
  File "/Users/ad/workSpace/UCSC/nanoPore/Chiron/chiron/utils/unix_time.py", line 23, in unix_time
    function(*args, **kwargs)
  File "/Users/ad/workSpace/UCSC/nanoPore/Chiron/chiron/chiron_eval.py", line 147, in evaluation
    saver = tf.train.Saver()
  File "/Users/ad/workSpace/pythonEnv/Chiron_env/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1040, in __init__
    self.build()
  File "/Users/ad/workSpace/pythonEnv/Chiron_env/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1070, in build
    restore_sequentially=self._restore_sequentially)
  File "/Users/ad/workSpace/pythonEnv/Chiron_env/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 675, in build
    restore_sequentially, reshape)
  File "/Users/ad/workSpace/pythonEnv/Chiron_env/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 402, in _AddRestoreOps
    tensors = self.restore_op(filename_tensor, saveable, preferred_shard)
  File "/Users/ad/workSpace/pythonEnv/Chiron_env/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 242, in restore_op
    [spec.tensor.dtype])[0])
  File "/Users/ad/workSpace/pythonEnv/Chiron_env/lib/python2.7/site-packages/tensorflow/python/ops/gen_io_ops.py", line 668, in restore_v2
    dtypes=dtypes, name=name)
  File "/Users/ad/workSpace/pythonEnv/Chiron_env/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 763, in apply_op
    op_def=op_def)
  File "/Users/ad/workSpace/pythonEnv/Chiron_env/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2327, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "/Users/ad/workSpace/pythonEnv/Chiron_env/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1226, in __init__
    self._traceback = _extract_stack()

NotFoundError (see above for traceback): Key block1dilate_layer1/dilate_branch/filter_branch/filter/weights not found in checkpoint
     [[Node: save/RestoreV2_12 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_save/Const_0, save/RestoreV2_12/tensor_names, save/RestoreV2_12/shape_and_slices)]]

entry.py.call.error.out.txt

haotianteng commented 6 years ago

Hi Andy, If you want to use the up-to-date version of Chiron please download the clean-code branch. git checkout clean-code should do the job.

There is also a new model in this branch, which should work with Tensorflow 1.3.0. But for other versions of Tensorflow, you have to train your model using that version of Tensorflow.

Another way is to freeze the model using the chiron/export_test.py, and run the froze model with chiron/load_run_test.py, but these scripts are still under development.

AEDWIP commented 6 years ago

Hi Lee

It looks like my virtualenv is using the correct version of tensorflow. It looks like there a lot of new development going on. Maybe I'll wait till clean-code branch gets merged back. I would prefer to use python3 and a newer version of tensorflow.

(Chiron_env) $ python --version Python 2.7.10 (Chiron_env) $ python -c 'import tensorflow as tf; print tf.version' 1.0.1 (Chiron_env) $

Kind regards

Andy

AEDWIP commented 6 years ago

Hi Haotianteg

I am happy to train a new model. Is your training data set available?

Kind regards

Andy

haotianteng commented 6 years ago

Clean_code branch is not ready, please wait when it merge to master.

haotianteng commented 6 years ago

Hi Andy, I have made a patch in the clean_code branch, just tested it on Tensorflow 1.3 Cuda8 and Tensorflow 1.5 Cuda9, all work fine now. If there is still problem you can continue reply under this thread.

Welcome for contributing, especially on training new models.

Thanks Teng

AEDWIP commented 6 years ago

Thanks

andy

From: Lee Yam Keng notifications@github.com Reply-To: haotianteng/Chiron <reply+00725a2fd0c699f20ca15a9bad472db6b7216ae96434b14092cf0000000116c66ab89 2a169ce12217c2e@reply.github.com> Date: Sunday, March 18, 2018 at 10:58 AM To: haotianteng/Chiron Chiron@noreply.github.com Cc: Andrew Davidson Andy@SantaCruzIntegration.com, Author author@noreply.github.com Subject: Re: [haotianteng/Chiron] unable to run 'chiron/entry.py call' (#30)

And Teng will send the download link to get the full dataset for training.