search

onnx / tensorflow-onnx

Convert TensorFlow, Keras, Tensorflow.js and Tflite models to ONNX
Apache License 2.0
2.32k stars 432 forks source link

ValueError: Graph has cycles #2246

Open ghost opened 1 year ago

ghost commented 1 year ago

Describe the bug 2023-09-22 13:51:09.484965: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used. 2023-09-22 13:51:09.533379: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used. 2023-09-22 13:51:09.533992: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. To enable the following instructions: AVX2 AVX512F FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags. 2023-09-22 13:51:10.343404: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT /home/studio-lab-user/.conda/envs/studiolab/lib/python3.9/runpy.py:127: RuntimeWarning: 'tf2onnx.convert' found in sys.modules after import of package 'tf2onnx', but prior to execution of 'tf2onnx.convert'; this may result in unpredictable behaviour warn(RuntimeWarning(msg)) 2023-09-22 13:51:11,142 - WARNING - '--tag' not specified for saved_model. Using --tag serve 2023-09-22 13:51:26,497 - INFO - Fingerprint not found. Saved model loading will continue. 2023-09-22 13:51:26,572 - INFO - Signatures found in model: [serving_default]. 2023-09-22 13:51:26,572 - WARNING - '--signature_def' not specified, using first signature: serving_default 2023-09-22 13:51:26,576 - INFO - Output names: ['detection_attributes', 'detection_boxes', 'detection_classes', 'detection_masks', 'detection_scores', 'image_info', 'num_detections'] 2023-09-22 13:51:26,576 - WARNING - Could not search for non-variable resources. Concrete function internal representation may have changed. WARNING:tensorflow:Issue encountered when serializing global_step. Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore. This operation is not supported when eager execution is enabled. 2023-09-22 13:51:26,825 - WARNING - Issue encountered when serializing global_step. Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore. This operation is not supported when eager execution is enabled. WARNING:tensorflow:Issue encountered when serializing variables. Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore. This operation is not supported when eager execution is enabled. 2023-09-22 13:51:26,827 - WARNING - Issue encountered when serializing variables. Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore. This operation is not supported when eager execution is enabled. WARNING:tensorflow:Issue encountered when serializing trainable_variables. Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore. This operation is not supported when eager execution is enabled. 2023-09-22 13:51:26,827 - WARNING - Issue encountered when serializing trainable_variables. Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore. This operation is not supported when eager execution is enabled. 2023-09-22 13:51:26.829350: I tensorflow/core/grappler/devices.cc:66] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 0 2023-09-22 13:51:26.829510: I tensorflow/core/grappler/clusters/single_machine.cc:357] Starting new session 2023-09-22 13:51:51.039397: I tensorflow/core/grappler/devices.cc:66] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 0 2023-09-22 13:51:51.039630: I tensorflow/core/grappler/clusters/single_machine.cc:357] Starting new session 2023-09-22 13:52:00,923 - INFO - Using tensorflow=2.13.0, onnx=1.14.1, tf2onnx=1.15.1/37820d 2023-09-22 13:52:00,923 - INFO - Using opset <onnx, 15> 2023-09-22 13:52:05,401 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,405 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,408 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,409 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,410 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,411 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,412 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,413 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,414 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,414 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,415 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,416 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,420 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,424 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,428 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,428 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,430 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,434 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,436 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,440 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,442 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,444 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,448 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,451 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,454 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,460 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,471 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,475 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,476 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,480 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,486 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,497 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,501 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,502 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,506 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,508 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,512 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,516 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,520 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,524 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,525 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,529 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,531 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,532 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,536 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,538 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,545 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,546 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,550 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,551 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,552 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,553 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,555 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,556 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,558 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,561 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,563 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,567 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,568 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,570 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,571 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,574 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,578 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,581 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,582 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,583 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,587 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,588 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,591 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,603 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,607 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,611 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,613 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,615 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,616 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,620 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,624 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,625 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,626 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,627 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,631 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,632 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,633 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,634 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,637 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,640 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,650 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,651 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,652 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,656 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,657 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,663 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,674 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,680 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,691 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,696 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,698 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,701 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,703 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,707 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,712 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,716 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,721 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,726 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,727 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,729 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,732 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,732 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,733 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,735 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,738 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,742 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,743 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,745 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,746 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,748 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,751 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,755 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,756 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,758 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,762 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,762 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,763 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,765 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,770 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,776 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,779 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,780 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,782 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05,789 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:05.817106: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:375] MLIR V1 optimization pass is not enabled 2023-09-22 13:52:05,823 - INFO - Computed 1 values for constant folding 2023-09-22 13:52:05,843 - INFO - folding node using tf type=ExpandDims, name=ExpandDims 2023-09-22 13:52:05,856 - INFO - Computed 1 values for constant folding 2023-09-22 13:52:05,875 - INFO - folding node using tf type=ExpandDims, name=ExpandDims 2023-09-22 13:52:05,890 - INFO - Computed 1 values for constant folding 2023-09-22 13:52:05,920 - INFO - folding node using tf type=ExpandDims, name=ExpandDims 2023-09-22 13:52:05,941 - INFO - Computed 1 values for constant folding 2023-09-22 13:52:05,962 - INFO - folding node using tf type=ExpandDims, name=ExpandDims 2023-09-22 13:52:05,974 - INFO - Computed 1 values for constant folding 2023-09-22 13:52:05,994 - INFO - folding node using tf type=ExpandDims, name=ExpandDims 2023-09-22 13:52:05,996 - INFO - Computed 0 values for constant folding 2023-09-22 13:52:06,647 - INFO - Computed 2 values for constant folding 2023-09-22 13:52:09,881 - INFO - folding node using tf type=Tile, name=multilevel_crop_and_resize/Tile 2023-09-22 13:52:09,882 - INFO - folding node using tf type=Tile, name=multilevel_crop_and_resize_1/Tile Traceback (most recent call last): File "/home/studio-lab-user/.conda/envs/studiolab/lib/python3.9/runpy.py", line 197, in _run_module_as_main return _run_code(code, main_globals, None, File "/home/studio-lab-user/.conda/envs/studiolab/lib/python3.9/runpy.py", line 87, in _run_code exec(code, run_globals) File "/home/studio-lab-user/.conda/envs/studiolab/lib/python3.9/site-packages/tf2onnx/convert.py", line 714, in <module> main() File "/home/studio-lab-user/.conda/envs/studiolab/lib/python3.9/site-packages/tf2onnx/convert.py", line 273, in main model_proto, _ = _convert_common( File "/home/studio-lab-user/.conda/envs/studiolab/lib/python3.9/site-packages/tf2onnx/convert.py", line 168, in _convert_common g = process_tf_graph(tf_graph, const_node_values=const_node_values, File "/home/studio-lab-user/.conda/envs/studiolab/lib/python3.9/site-packages/tf2onnx/tfonnx.py", line 464, in process_tf_graph g = process_graphs(main_g, subgraphs, custom_op_handlers, inputs_as_nchw, outputs_as_nchw, continue_on_error, File "/home/studio-lab-user/.conda/envs/studiolab/lib/python3.9/site-packages/tf2onnx/tfonnx.py", line 513, in process_graphs fg = process_parsed_graph(g, custom_op_handlers, inputs_as_nchw, outputs_as_nchw, continue_on_error, File "/home/studio-lab-user/.conda/envs/studiolab/lib/python3.9/site-packages/tf2onnx/tfonnx.py", line 641, in process_parsed_graph topological_sort(g, continue_on_error) File "/home/studio-lab-user/.conda/envs/studiolab/lib/python3.9/site-packages/tf2onnx/tfonnx.py", line 359, in topological_sort g.topological_sort(ops) File "/home/studio-lab-user/.conda/envs/studiolab/lib/python3.9/site-packages/tf2onnx/graph.py", line 1075, in topological_sort _push_stack(stack, node, in_stack) File "/home/studio-lab-user/.conda/envs/studiolab/lib/python3.9/site-packages/tf2onnx/graph.py", line 1032, in _push_stack raise ValueError('Graph has cycles, node.name=%r.' % ops[node].name) ValueError: Graph has cycles, node.name='while_1_loop'.

Urgency Very urgent. 25.09.2023

System information -" Ubuntu" "20.04.3 LTS (Focal Fossa)"

To Reproduce

python -m tf2onnx.convert --saved-model ./x/ --output model.onnx

fatcat-z commented 1 year ago

Is that possible for you to share the model or part of it so I can do a local debug?