Problems with converted model

lp55 commented 5 years ago

Hi,

I'm trying to convert a tiny yolo v3 trained in a custom dataset with a single class to coreml. I managed to convert to tensorflow using this DW2TF tool (just some deprecated warnings), but when I try converting the tensorflow graph to coreml using tf-coreml, I got the following errors:

Graph Loaded. Collecting all the 'Const' ops from the graph, by running it.... Traceback (most recent call last): File "/home/dock/.conda/envs/custommodelapp_dw/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1334, in _do_call return fn(*args) File "/home/dock/.conda/envs/custommodelapp_dw/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1319, in _run_fn options, feed_dict, fetch_list, target_list, run_metadata) File "/home/dock/.conda/envs/custommodelapp_dw/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun run_metadata) tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value network/convolutional11/BatchNorm/moving_variance [[{{node network/convolutional11/BatchNorm/moving_variance/read}}]] [[{{node network/convolutional4/BatchNorm/moving_variance/Initializer/ones}}]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "ts_coreml.py", line 9, in image_scale = 1 / 255.0) File "/home/dock/.conda/envs/custommodelapp_dw/lib/python3.6/site-packages/tfcoreml/_tf_coreml_converter.py", line 586, in convert custom_conversion_functions=custom_conversion_functions) File "/home/dock/.conda/envs/custommodelapp_dw/lib/python3.6/site-packages/tfcoreml/_tf_coreml_converter.py", line 243, in _convert_pb_to_mlmodel tensors_evaluated = sess.run(tensors, feed_dict=input_feed_dict) File "/home/dock/.conda/envs/custommodelapp_dw/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 929, in run run_metadata_ptr) File "/home/dock/.conda/envs/custommodelapp_dw/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1152, in _run feed_dict_tensor, options, run_metadata) File "/home/dock/.conda/envs/custommodelapp_dw/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1328, in _do_run run_metadata) File "/home/dock/.conda/envs/custommodelapp_dw/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1348, in _do_call raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value network/convolutional11/BatchNorm/moving_variance [[node network/convolutional11/BatchNorm/moving_variance/read (defined at /home/dock/.conda/envs/custommodelapp_dw/lib/python3.6/site-packages/tfcoreml/_tf_coreml_converter.py:153) ]] [[node network/convolutional4/BatchNorm/moving_variance/Initializer/ones (defined at /home/dock/.conda/envs/custommodelapp_dw/lib/python3.6/site-packages/tfcoreml/_tf_coreml_converter.py:153) ]]

Caused by op 'network/convolutional11/BatchNorm/moving_variance/read', defined at: File "ts_coreml.py", line 9, in image_scale = 1 / 255.0) File "/home/dock/.conda/envs/custommodelapp_dw/lib/python3.6/site-packages/tfcoreml/_tf_coreml_converter.py", line 586, in convert custom_conversion_functions=custom_conversion_functions) File "/home/dock/.conda/envs/custommodelapp_dw/lib/python3.6/site-packages/tfcoreml/_tf_coreml_converter.py", line 153, in _convert_pb_to_mlmodel tf.import_graph_def(gdef, name='') File "/home/dock/.conda/envs/custommodelapp_dw/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func return func(*args, **kwargs) File "/home/dock/.conda/envs/custommodelapp_dw/lib/python3.6/site-packages/tensorflow/python/framework/importer.py", line 442, in import_graph_def _ProcessNewOps(graph) File "/home/dock/.conda/envs/custommodelapp_dw/lib/python3.6/site-packages/tensorflow/python/framework/importer.py", line 235, in _ProcessNewOps for new_op in graph._add_new_tf_operations(compute_devices=False): # pylint: disable=protected-access File "/home/dock/.conda/envs/custommodelapp_dw/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3433, in _add_new_tf_operations for c_op in c_api_util.new_tf_operations(self) File "/home/dock/.conda/envs/custommodelapp_dw/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3433, in for c_op in c_api_util.new_tf_operations(self) File "/home/dock/.conda/envs/custommodelapp_dw/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3325, in _create_op_from_tf_operation ret = Operation(c_op, self) File "/home/dock/.conda/envs/custommodelapp_dw/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1801, in init self._traceback = tf_stack.extract_stack()

FailedPreconditionError (see above for traceback): Attempting to use uninitialized value network/convolutional11/BatchNorm/moving_variance [[node network/convolutional11/BatchNorm/moving_variance/read (defined at /home/dock/.conda/envs/custommodelapp_dw/lib/python3.6/site-packages/tfcoreml/_tf_coreml_converter.py:153) ]] [[node network/convolutional4/BatchNorm/moving_variance/Initializer/ones (defined at /home/dock/.conda/envs/custommodelapp_dw/lib/python3.6/site-packages/tfcoreml/_tf_coreml_converter.py:153) ]]

I used this simple code to try to make this conversion:

import tfcoreml as tf_converter

tf_converter.convert(tf_model_path = 'data/custommodel_tiny.pb', mlmodel_path = 'data/custommodel_tiny.mlmodel', output_feature_names = ['detect_1', 'detect_2'], is_bgr = False, input_name_shape_dict = {'input1' : [1, 608, 608, 3]}, image_input_names = ['input1'], image_scale = 1 / 255.0)

I had success with tf-coreml in other conversions, and as the errors point to problems in preconditions, I think there might be something with the darknet -> tensorflow conversion that didn't go so well. I viewed the produced pb on tensorboard: convertDW2F

 Any ideas?

shenyingying commented 5 years ago

@lp55 @jinyu121 have you resolved the problem, guided by readme, i success convert yolov3.weight to yolov3.pb , does the .pb file input is input the output is : or as the darkflow but when i use the .pb file it throw the error has no node_name yolov3/net1:0

zdmwang commented 5 years ago

I have the same error。Did you have resolved the problem? If you have already solved this problem, please contact me. My email address is: xdzengdm@163.com. Thank you..

lp55 commented 5 years ago

I never solved this problem. I started using turicreate to create coreml models. It's not a good framework but it's the only one I got it to work with.

zdmwang commented 5 years ago

Ok，thank you very much.

                            469143253

                                邮箱：469143253@qq.com

    Signature is customized by Netease Mail Master

On 07/20/2019 00:36, lp55 wrote: I never solved this problem. I started using turicreate to create coreml models. It's not a good framework but it's the only one I got it to work with.

—You are receiving this because you commented.Reply to this email directly, view it on GitHub, or mute the thread. [ { "@context": "http://schema.org", "@type": "EmailMessage", "potentialAction": { "@type": "ViewAction", "target": "https://github.com/jinyu121/DW2TF/issues/21?email_source=notifications\u0026email_token=AINPOBFPVRWORO2QVQUUA53QAHUPHA5CNFSM4HPQ5SKKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2MEDUI#issuecomment-513294801", "url": "https://github.com/jinyu121/DW2TF/issues/21?email_source=notifications\u0026email_token=AINPOBFPVRWORO2QVQUUA53QAHUPHA5CNFSM4HPQ5SKKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2MEDUI#issuecomment-513294801", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { "@type": "Organization", "name": "GitHub", "url": "https://github.com" } } ]

zdmwang commented 5 years ago

hello，I have solved this problem. The reason is that the parameters are not aligned. I changed the self.offset = 16 on line 39 of read.py in utils to self.offset = 20 and the problem is solved . You can read the parameters of each layer and set the value of self.offset.

                            469143253

                                邮箱：469143253@qq.com

    Signature is customized by Netease Mail Master

On 07/22/2019 14:14, zdmwang wrote: Ok，thank you very much.

                            469143253

                                邮箱：469143253@qq.com

    Signature is customized by Netease Mail Master

On 07/20/2019 00:36, lp55 wrote: I never solved this problem. I started using turicreate to create coreml models. It's not a good framework but it's the only one I got it to work with.

—You are receiving this because you commented.Reply to this email directly, view it on GitHub, or mute the thread. [ { "@context": "http://schema.org", "@type": "EmailMessage", "potentialAction": { "@type": "ViewAction", "target": "https://github.com/jinyu121/DW2TF/issues/21?email_source=notifications\u0026email_token=AINPOBFPVRWORO2QVQUUA53QAHUPHA5CNFSM4HPQ5SKKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2MEDUI#issuecomment-513294801", "url": "https://github.com/jinyu121/DW2TF/issues/21?email_source=notifications\u0026email_token=AINPOBFPVRWORO2QVQUUA53QAHUPHA5CNFSM4HPQ5SKKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2MEDUI#issuecomment-513294801", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { "@type": "Organization", "name": "GitHub", "url": "https://github.com" } } ]

sjain-stanford commented 5 years ago

@zdmwang The offset was initially hard-coded to 16, but this was later fixed in cbb8516. Are you using the latest clone of DW2TF? If not, please do a git pull.

@lp55 There was another fix (for #30) that impacts tiny yolo-v3 or yolo-v3. It is released in DW2TF-1.2. Please fetch this and re-run.

codeseeking commented 4 years ago

@lp55 @jinyu121 have you resolved the problem, guided by readme, i success convert yolov3.weight to yolov3.pb , does the .pb file input is the output is : or as the darkflow but when i use the .pb file it throw the error has no node_name yolov3/net1:0

@lp55 @jinyu121 have you resolved the problem, guided by readme, i success convert yolov3.weight to yolov3.pb , does the .pb file input is the output is : or as the darkflow but when i use the .pb file it throw the error has no node_name yolov3/net1:0

Have you addressed this problem,I meet the same problem

zdmwang commented 4 years ago

从你的输出来结果来看，graph的输入节点名字确实是"yolov3/net1:0"，建议你将graph中的节点名称输出查看一下。

------------------ Original ------------------ From: "mhealth";notifications@github.com; Send time: Tuesday, Jan 14, 2020 10:17 AM To: "jinyu121/DW2TF"DW2TF@noreply.github.com; Cc: "曾丹梦"469143253@qq.com; "Mention"mention@noreply.github.com; Subject: Re: [jinyu121/DW2TF] Problems with converted model (#21)

@lp55 @jinyu121 have you resolved the problem, guided by readme, i success convert yolov3.weight to yolov3.pb , does the .pb file input is

the output is :

or as the darkflow but when i use the .pb file it throw the error has no node_name yolov3/net1:0

@lp55 @jinyu121 have you resolved the problem, guided by readme, i success convert yolov3.weight to yolov3.pb , does the .pb file input is

the output is :

or as the darkflow but when i use the .pb file it throw the error has no node_name yolov3/net1:0

Have you addressed this problem,I meet the same problem

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

jinyu121 / DW2TF

Problems with converted model #21