search

melobio / LOGO

MIT License
19 stars 3 forks source link

CADD prediction-can't parse single sample #3

Open Shenying71 opened 3 years ago

Shenying71 commented 3 years ago

Hello,

When I ran the model “02_cadd_classification_transformer_tfrecord.py” for CADD data, I used the following data and ran into error messages: can’t parse serialized example. Can you provide help? I would like to apply this process to predict pathogenic effects of somatic variants. Or can you provide revised codes and an example data?

train_slice_files = [ '/rsrch3/home/surgonc_rsrch/sfang/LOGO/LOGO-master/05_LOGO_Variant_Prioritization/CADD/GRCh37/SNVS1/humanDerived_SNVs_gram_5_stride_5_slice_100000_100000_train.tfrecord','/rsrch3/home/surgonc_rsrch/sfang/LOGO/LOGO-master/05_LOGO_Variant_Prioritization/CADD/GRCh37/SNVS1/humanDerived_SNVs_gram_5_stride_5_slice_200000_200000_train.tfrecord' ] valid_slice_files = [ '/rsrch3/home/surgonc_rsrch/sfang/LOGO/LOGO-master/05_LOGO_Variant_Prioritization/CADD/GRCh37/SNVS1/humanDerived_SNVs_gram_5_stride_5_slice_100000_100000_valid.tfrecord' ] test_slice_files = [ '/rsrch3/home/surgonc_rsrch/sfang/LOGO/LOGO-master/05_LOGO_Variant_Prioritization/CADD/GRCh37/SNVS1/humanDerived_SNVs_gram_5_stride_5_slice_100000_100000_test.tfrecord' ]

        train_total_size =180000
        valid_total_size = 5000
        test_total_size = 5000

I ran into the following errors:

Train on 175 steps, validate on 2 steps Epoch 1/100 2021-10-25 21:11:09.434153: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: alt_seq. Can't parse serialized Example. 2021-10-25 21:11:09.434270: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: alt_seq. Can't parse serialized Example. Traceback (most recent call last): File "02_cadd_classification_transformer_tfrecord.py", line 278, in verbose=1) File "/risapps/rhel7/python/3.7.3/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training.py", line 727, in fit 2021-10-25 21:11:09.434995: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: alt_type. Can't parse serialized Example. 2021-10-25 21:11:09.435078: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: seq. Can't parse serialized Example. 2021-10-25 21:11:09.435134: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: alt_seq. Can't parse serialized Example. 2021-10-25 21:11:09.435209: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: alt_type. Can't parse serialized Example. 2021-10-25 21:11:09.435261: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: alt_seq. Can't parse serialized Example. use_multiprocessing=use_multiprocessing) File "/risapps/rhel7/python/3.7.3/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_arrays.py", line 675, in fit 2021-10-25 21:11:09.435613: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: alt_type. Can't parse serialized Example. 2021-10-25 21:11:09.435697: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: seq. Can't parse serialized Example. steps_name='steps_per_epoch') File "/risapps/rhel7/python/3.7.3/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_arrays.py", line 300, in model_iteration 2021-10-25 21:11:09.436015: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: alt_seq. Can't parse serialized Example. batch_outs = f(actual_inputs) File "/risapps/rhel7/python/3.7.3/lib/python3.7/site-packages/tensorflow_core/python/keras/backend.py", line 3476, in call 2021-10-25 21:11:09.436259: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: alt_seq. Can't parse serialized Example. 2021-10-25 21:11:09.436336: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: alt_seq. Can't parse serialized Example. run_metadata=self.run_metadata) File "/risapps/rhel7/python/3.7.3/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1472, in call 2021-10-25 21:11:09.437045: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: alt_type. Can't parse serialized Example. 2021-10-25 21:11:09.437105: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: alt_type. Can't parse serialized Example. 2021-10-25 21:11:09.437173: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: alt_seq. Can't parse serialized Example. 2021-10-25 21:11:09.437225: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: seq. Can't parse serialized Example. run_metadata_ptr) tensorflow.python.framework.errors_impl.InvalidArgumentError: {{function_node __inference_Dataset_map_single_example_parser_1272}} Key: alt_seq. Can't parse serialized Example. [[{{node ParseSingleExample/ParseSingleExample}}]] [[IteratorGetNext]] 2021-10-25 21:11:09.452867: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: seq. Can't parse serialized Example. 2021-10-25 21:11:09.452934: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: alt_type. Can't parse serialized Example.