agentmorris
commented
1 year ago I'm not an expert here, but it appears that a number of operations required by our detector are not supported by TFLite:
https://www.tensorflow.org/lite/guide/faq#why_doesnt_my_model_convert
We may be able to improve convertibility by changing the way we freeze our model the next time we re-train MegaDetector; it appears to depend on TF's timeline for integrating various operations into TFLite. That said, TFLite is designed to be a subset of TF, so it may never quite support all the required operations.
FWIW, others have had success in similar scenarios by using MegaDetector to generate bounding boxes on your images (and/or on public images from http://lila.science that are representative of your ecosystem), and using those bounding boxes to train the largest model that fits your compute budget. You may get the benefit of compression here and the benefit of fine-tuning to your ecosystem(s).
Hope that helps, sorry there's not an easy conversion path that I'm aware of!
(Comment originally posted by agentmorris)
I am attempting to port the frozen .pb file to a TFLite model for use with the Raspberry Pi and the Google Coral TPU USB Coprocessor and ran into issues running the conversion. I am new to TensorFlow and, while I'm a Software Engineer, I'm using this as a user more than a developer, so bear with me here.
The command I run and samples of the, very verbose, output are below. I know this may not be at the top of your priority list, but if it helps motivation, if this works out we could use MegaDetector on camera traps in the field to detect animals in real-time in the field, like for a predator detector I'm looking for for our farm. I'm able to crunch a detection in 45 seconds on my CPU but am eager to see what the Coral TPU can do. Thanks in advance for any help you're able to provide and I'm happy to roll up my sleeves and dig in if I can get a nudge in the right direction.
Here's the command I run: tflite_convert --output_file=md_v4.1.0.pb.tflite --graph_def_file md_v4.1.0.pb --input_arrays=image_tensor --output_arrays=detection_boxes,detection_scores,detection_classes --input_shapes=1,640,480,3 --enable_v1_converter
Traceback (most recent call last): File "c:\anaconda3\envs\py36\lib\runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "c:\anaconda3\envs\py36\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "C:\anaconda3\envs\py36\Scripts\tflite_convert.exe__main__.py", line 7, in File "c:\anaconda3\envs\py36\lib\site-packages\tensorflow\lite\python\tflite_convert.py", line 442, in main app.run(main=run_main, argv=sys.argv[:1]) File "c:\anaconda3\envs\py36\lib\site-packages\tensorflow\python\platform\app.py", line 125, in run _sys.exit(main(argv)) File "c:\anaconda3\envs\py36\lib\site-packages\tensorflow\lite\python\tflite_convert.py", line 438, in run_main _convert_model(tflite_flags) File "c:\anaconda3\envs\py36\lib\site-packages\tensorflow\lite\python\tflite_convert.py", line 191, in _convert_model output_data = converter.convert() File "c:\anaconda3\envs\py36\lib\site-packages\tensorflow\lite\python\lite.py", line 455, in convert **converter_kwargs) File "c:\anaconda3\envs\py36\lib\site-packages\tensorflow\lite\python\convert.py", line 442, in toco_convert_impl input_data.SerializeToString()) File "c:\anaconda3\envs\py36\lib\site-packages\tensorflow\lite\python\convert.py", line 205, in toco_convert_protos "TOCO failed. See console for info.\n%s\n%s\n" % (stdout, stderr)) tensorflow.lite.python.convert.ConverterError: TOCO failed. See console for info. 2020-12-10 21:12:31.516113: I tensorflow/lite/toco/import_tensorflow.cc:1324] Converting unsupported operation: TensorArrayV3 2020-12-10 21:12:31.516570: E tensorflow/core/framework/op_kernel.cc:1325] OpKernel ('op: "WrapDatasetVariant" device_type: "CPU"') for unknown op: WrapDatasetVariant 2020-12-10 21:12:31.516902: E tensorflow/core/framework/op_kernel.cc:1325] OpKernel ('op: "WrapDatasetVariant" device_type: "GPU" host_memory_arg: "input_handle" host_memory_arg: "output_handle"') for unknown op: WrapDatasetVariant 2020-12-10 21:12:31.517337: E tensorflow/core/framework/op_kernel.cc:1325] OpKernel ('op: "UnwrapDatasetVariant" device_type: "CPU"') for unknown op: UnwrapDatasetVariant 2020-12-10 21:12:31.517673: E tensorflow/core/framework/op_kernel.cc:1325] OpKernel ('op: "UnwrapDatasetVariant" device_type: "GPU" host_memory_arg: "input_handle" host_memory_arg: "output_handle"') for unknown op: UnwrapDatasetVariant 2020-12-10 21:12:31.518258: I tensorflow/lite/toco/import_tensorflow.cc:1373] Unable to determine output type for op: TensorArrayV3 2020-12-10 21:12:31.518503: I tensorflow/lite/toco/import_tensorflow.cc:1324] Converting unsupported operation: TensorArrayScatterV3 2020-12-10 21:12:31.518752: I tensorflow/lite/toco/import_tensorflow.cc:1373] Unable to determine output type for op: TensorArrayScatterV3 2020-12-10 21:12:31.519004: I tensorflow/lite/toco/import_tensorflow.cc:1324] Converting unsupported operation: TensorArrayV3 2020-12-10 21:12:31.519201: I tensorflow/lite/toco/import_tensorflow.cc:1373] Unable to determine output type for op: TensorArrayV3 2020-12-10 21:12:31.519404: I tensorflow/lite/toco/import_tensorflow.cc:1324] Converting unsupported operation: TensorArrayV3 2020-12-10 21:12:31.519606: I tensorflow/lite/toco/import_tensorflow.cc:1373] Unable to determine output type for op: TensorArrayV3 2020-12-10 21:12:31.519863: I tensorflow/lite/toco/import_tensorflow.cc:1324] Converting unsupported operation: Enter 2020-12-10 21:12:31.520066: I tensorflow/lite/toco/import_tensorflow.cc:1373] Unable to determine output type for op: Enter
..........
2020-12-10 21:12:31.887365: I tensorflow/lite/toco/import_tensorflow.cc:1373] Unable to determine output type for op: TensorArrayGatherV3 2020-12-10 21:12:34.795091: I tensorflow/lite/toco/graph_transformations/graph_transformations.cc:39] Before Removing unused ops: 8696 operators, 17295 arrays (0 quantized) 2020-12-10 21:12:37.623228: I tensorflow/lite/toco/graph_transformations/graph_transformations.cc:39] After Removing unused ops pass 1: 8662 operators, 17241 arrays (0 quantized) 2020-12-10 21:12:41.875052: I tensorflow/lite/toco/graph_transformations/graph_transformations.cc:39] Before general graph transformations: 8662 operators, 17241 arrays (0 quantized) 2020-12-10 21:12:42.990606: I tensorflow/lite/toco/graph_transformations/identify_dilated_conv.cc:220] Replaced sub-network with Dilated Conv2D op outputting "FirstStageFeatureExtractor/InceptionResnetV2/InceptionResnetV2/Repeat_1/block17_1/Conv2d_1x1/Conv2D". 2020-12-10 21:12:43.114359: I tensorflow/lite/toco/graph_transformations/identify_dilated_conv.cc:220] Replaced sub-network with Dilated Conv2D op outputting "FirstStageFeatureExtractor/InceptionResnetV2/InceptionResnetV2/Repeat_1/block17_2/Conv2d_1x1/Conv2D". 2020-12-10 21:12:43.249639: I tensorflow/lite/toco/graph_transformations/identify_dilated_conv.cc:220] Replaced sub-network with Dilated Conv2D op outputting "FirstStageFeatureExtractor/InceptionResnetV2/InceptionResnetV2/Repeat_1/block17_3/Conv2d_1x1/Conv2D". 2020-12-10 21:12:43.371587: I tensorflow/lite/toco/graph_transformations/identify_dilated_conv.cc:220] Replaced sub-network with Dilated Conv2D op outputting "FirstStageFeatureExtractor/InceptionResnetV2/InceptionResnetV2/Repeat_1/block17_4/Conv2d_1x1/Conv2D". 2020-12-10 21:12:43.490386: I tensorflow/lite/toco/graph_transformations/identify_dilated_conv.cc:220] Replaced sub-network with Dilated Conv2D op outputting "FirstStageFeatureExtractor/InceptionResnetV2/InceptionResnetV2/Repeat_1/block17_5/Conv2d_1x1/Conv2D". 2020-12-10 21:12:43.610138: I tensorflow/lite/toco/graph_transformations/identify_dilated_conv.cc:220] Replaced sub-network with Dilated Conv2D op outputting "FirstStageFeatureExtractor/InceptionResnetV2/InceptionResnetV2/Repeat_1/block17_6/Conv2d_1x1/Conv2D". 2020-12-10 21:12:43.729914: I tensorflow/lite/toco/graph_transformations/identify_dilated_conv.cc:220] Replaced sub-network with Dilated Conv2D op outputting "FirstStageFeatureExtractor/InceptionResnetV2/InceptionResnetV2/Repeat_1/block17_7/Conv2d_1x1/Conv2D". 2020-12-10 21:12:43.845808: I tensorflow/lite/toco/graph_transformations/identify_dilated_conv.cc:220] Replaced sub-network with Dilated Conv2D op outputting "FirstStageFeatureExtractor/InceptionResnetV2/InceptionResnetV2/Repeat_1/block17_8/Conv2d_1x1/Conv2D". 2020-12-10 21:12:43.961710: I tensorflow/lite/toco/graph_transformations/identify_dilated_conv.cc:220] Replaced sub-network with Dilated Conv2D op outputting "FirstStageFeatureExtractor/InceptionResnetV2/InceptionResnetV2/Repeat_1/block17_9/Conv2d_1x1/Conv2D". 2020-12-10 21:12:44.079812: I tensorflow/lite/toco/graph_transformations/identify_dilated_conv.cc:220] Replaced sub-network with Dilated Conv2D op outputting "FirstStageFeatureExtractor/InceptionResnetV2/InceptionResnetV2/Repeat_1/block17_10/Conv2d_1x1/Conv2D". 2020-12-10 21:12:44.198136: I tensorflow/lite/toco/graph_transformations/identify_dilated_conv.cc:220] Replaced sub-network with Dilated Conv2D op outputting "FirstStageFeatureExtractor/InceptionResnetV2/InceptionResnetV2/Repeat_1/block17_11/Conv2d_1x1/Conv2D". 2020-12-10 21:12:44.316202: I tensorflow/lite/toco/graph_transformations/identify_dilated_conv.cc:220] Replaced sub-network with Dilated Conv2D op outputting "FirstStageFeatureExtractor/InceptionResnetV2/InceptionResnetV2/Repeat_1/block17_12/Conv2d_1x1/Conv2D". 2020-12-10 21:12:44.430921: I tensorflow/lite/toco/graph_transformations/identify_dilated_conv.cc:220] Replaced sub-network with Dilated Conv2D op outputting "FirstStageFeatureExtractor/InceptionResnetV2/InceptionResnetV2/Repeat_1/block17_13/Conv2d_1x1/Conv2D". 2020-12-10 21:12:44.545152: I tensorflow/lite/toco/graph_transformations/identify_dilated_conv.cc:220] Replaced sub-network with Dilated Conv2D op outputting "FirstStageFeatureExtractor/InceptionResnetV2/InceptionResnetV2/Repeat_1/block17_14/Conv2d_1x1/Conv2D". 2020-12-10 21:12:44.657000: I tensorflow/lite/toco/graph_transformations/identify_dilated_conv.cc:220] Replaced sub-network with Dilated Conv2D op outputting "FirstStageFeatureExtractor/InceptionResnetV2/InceptionResnetV2/Repeat_1/block17_15/Conv2d_1x1/Conv2D". 2020-12-10 21:12:44.767624: I tensorflow/lite/toco/graph_transformations/identify_dilated_conv.cc:220] Replaced sub-network with Dilated Conv2D op outputting "FirstStageFeatureExtractor/InceptionResnetV2/InceptionResnetV2/Repeat_1/block17_16/Conv2d_1x1/Conv2D". 2020-12-10 21:12:44.878775: I tensorflow/lite/toco/graph_transformations/identify_dilated_conv.cc:220] Replaced sub-network with Dilated Conv2D op outputting "FirstStageFeatureExtractor/InceptionResnetV2/InceptionResnetV2/Repeat_1/block17_17/Conv2d_1x1/Conv2D". 2020-12-10 21:12:44.987341: I tensorflow/lite/toco/graph_transformations/identify_dilated_conv.cc:220] Replaced sub-network with Dilated Conv2D op outputting "FirstStageFeatureExtractor/InceptionResnetV2/InceptionResnetV2/Repeat_1/block17_18/Conv2d_1x1/Conv2D". 2020-12-10 21:12:45.098353: I tensorflow/lite/toco/graph_transformations/identify_dilated_conv.cc:220] Replaced sub-network with Dilated Conv2D op outputting "FirstStageFeatureExtractor/InceptionResnetV2/InceptionResnetV2/Repeat_1/block17_19/Conv2d_1x1/Conv2D". 2020-12-10 21:12:45.207493: I tensorflow/lite/toco/graph_transformations/identify_dilated_conv.cc:220] Replaced sub-network with Dilated Conv2D op outputting "FirstStageFeatureExtractor/InceptionResnetV2/InceptionResnetV2/Repeat_1/block17_20/Conv2d_1x1/Conv2D". 2020-12-10 21:12:45.215230: F tensorflow/lite/toco/graph_transformations/resolve_constant_slice.cc:59] Check failed: dim_size >= 1 (0 vs. 1)
Issue cloned from Microsoft/CameraTraps, original issue posted by RogerWebb on Dec 11, 2020.