search

tensorflow / models

Models and examples built with TensorFlow
Other
76.95k stars 45.79k forks source link

Unable to deploy ssd-fpn model to mobile #6084

Open AliceDinh opened 5 years ago

AliceDinh commented 5 years ago

System information

maxcrous commented 5 years ago

Thanks for creating a separate issue on this. @achowdhery and I went over the details of reproducing the error in #5298. They suspect it is a bug with Mul op, and will be looking into it in the next few days.

oopsodd commented 5 years ago

I used TF object detection API to train ssd_resnet_50_fpn_coco with a 50-classes dataset. Everything is okay with frozen model. The checkpoint was converted successfully. But the tflite model detect wrong class, bbox. All the output classes are the same (1 class). The tflite model takes 3s per image to inference on Galaxy S9 (same as the frozen model). Did Tflite support ssd_resnet_50_fpn_coco?

@AliceDinh Ban su dung mobilenet hay resnet?

AliceDinh commented 5 years ago

@oopsodd I used mobilenet but could not deploy to mobile, so you deploy resnet fpn to mobile successfully, right? So I have to convert my ssd-mobilenet-fpn frozen model to tflite model before deploy?

oopsodd commented 5 years ago

@AliceDinh Ban lam theo huong dan o day dung k, voi inference type la Float? https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/running_on_mobile_tensorflowlite.md Freeze model truoc bang export_tflite_ssd_graph.py Co ve nhu tflite van chua ho tro FPN dau :(

AliceDinh commented 5 years ago

@oopsodd To train model theo huong dan nay co: https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/running_locally.md Sau do thi export_inference_graph su dung script o link nay: https://github.com/tensorflow/models/blob/master/research/object_detection/export_inference_graph.py, thu duoc frozen_inference_graph.pb Cuoi cung, thi import frozen_inference_graph.pb vao app nay: https://github.com/tensorflow/tensorflow/tree/master/tensorflow/examples/android thi khong nhan dang duoc. P/s: frozen_inference_graph.pb của SSD-Mobilenet thi van duoc, nhưng ssd-fpn-mobilenet thi khong duoc

AliceDinh commented 5 years ago

The checkpoint was converted successfully.

@oopsodd You converted by using object_detection/export_tflite_ssd_graph.py or using tflite_convert of toco?

oopsodd commented 5 years ago

Ban dau minh tuong ban noi den tflite, minh convert sang frozen bang export-ssd-grah, sau do dung tiep TOCO. Theo dung huong dan convert sang tflite.

oopsodd commented 5 years ago

Ve frozen model export bang export-inference-graph, ben minh co nguoi deploy va report la la ssd resnet fpn detect duoc. Minh k truc tiep sua android camera demo app nen khong ro : (

AliceDinh commented 5 years ago

@oopsodd Yep, ban dau minh convert sang frozen bang: object_detection/export_inference_graph.py, sau đó dùng TOCO tflite_convert thì đang có lỗi tflite_convert - ValueError: Invalid tensors 'input' were found. Cho minh hoi trong khoi lenh convert thì --input_arrays và --output_arrays là gì?

oopsodd commented 5 years ago

Trong nay huong dan day du roi ma. https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/running_on_mobile_tensorflowlite.md Input, output giong example command trong ay luon.

thusinh1969 commented 5 years ago

Làm như này:

Convert into tflite-compatible pb. Lưu ý --max_detections:

python export_tflite_ssd_graph.py --input_type image_tensor --pipeline_config_path training_ssd_v2_config --trained_checkpoint_prefix ../output/model.ckpt-124960 --output_directory ../output/frozen_tflite/frozen_inference_graph.pb -add_postprocessing_op True --max_detections 10

--> Sinh ra 2 file tflite_graph.pb và tflite_graph.pbtxt

Sau đó make sure là chạy Tensorflow (hay Tensorflow GPU) bản 1.11 trở lên:

tflite_convert --output_file=./tflite/detect.tflite \ --graph_def_file=/home/nguyen/ssd/Foods/dataset/OD_food_non_food/ssd_model/output/frozen_tflite/frozen_inference_graph.pb/tflite_graph.pb \ --input_arrays=normalized_input_image_tensor \ --output_arrays='TFLite_Detection_PostProcess','TFLite_Detection_PostProcess:1','TFLite_Detection_PostProcess:2','TFLite_Detection_PostProcess:3' \ --input_shape=1,300,300,3 \ --allow_custom_ops

Ăn ngay :dango:

jvishnuvardhan commented 5 years ago

@AliceDinh Is this still an issue or was it resolved? Thanks!

AliceDinh commented 5 years ago

Hi @thusinh1969, Sorry I did not response to your suggestion early. I fail at your second step:

Traceback (most recent call last):
  File "c:\users\alice\anaconda3\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\alice\anaconda3\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\Alice\Anaconda3\Scripts\tflite_convert.exe\__main__.py", line 9, in <module>
  File "c:\users\alice\anaconda3\lib\site-packages\tensorflow\lite\python\tflite_convert.py", line 442, in main
    app.run(main=run_main, argv=sys.argv[:1])
  File "c:\users\alice\anaconda3\lib\site-packages\tensorflow\python\platform\app.py", line 40, in run
    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
  File "c:\users\alice\anaconda3\lib\site-packages\absl\app.py", line 300, in run
    _run_main(main, args)
  File "c:\users\alice\anaconda3\lib\site-packages\absl\app.py", line 251, in _run_main
    sys.exit(main(argv))
  File "c:\users\alice\anaconda3\lib\site-packages\tensorflow\lite\python\tflite_convert.py", line 438, in run_main
    _convert_model(tflite_flags)
  File "c:\users\alice\anaconda3\lib\site-packages\tensorflow\lite\python\tflite_convert.py", line 191, in _convert_model
    output_data = converter.convert()
  File "c:\users\alice\anaconda3\lib\site-packages\tensorflow\lite\python\lite.py", line 613, in convert
    **converter_kwargs)
  File "c:\users\alice\anaconda3\lib\site-packages\tensorflow\lite\python\convert.py", line 411, in toco_convert_graph_def
    input_data.SerializeToString())
  File "c:\users\alice\anaconda3\lib\site-packages\tensorflow\lite\python\convert.py", line 205, in toco_convert_protos
    "TOCO failed. See console for info.\n%s\n%s\n" % (stdout, stderr))
tensorflow.lite.python.convert.ConverterError: TOCO failed. See console for info.
2019-06-24 10:57:39.412451: I tensorflow/lite/toco/import_tensorflow.cc:1332] Converting unsupported operation: TFLite_Detection_PostProcess
2019-06-24 10:57:39.413056: E tensorflow/core/framework/op_kernel.cc:1336] OpKernel ('op: "NoOp" device_type: "CPU"') for unknown op: NoOp
2019-06-24 10:57:39.413289: E tensorflow/core/framework/op_kernel.cc:1336] OpKernel ('op: "NoOp" device_type: "GPU"') for unknown op: NoOp
2019-06-24 10:57:39.413539: E tensorflow/core/framework/op_kernel.cc:1336] OpKernel ('op: "_Send" device_type: "CPU"') for unknown op: _Send
2019-06-24 10:57:39.413796: E tensorflow/core/framework/op_kernel.cc:1336] OpKernel ('op: "_HostRecv" device_type: "CPU"') for unknown op: _HostRecv
2019-06-24 10:57:39.414028: E tensorflow/core/framework/op_kernel.cc:1336] OpKernel ('op: "_Send" device_type: "GPU"') for unknown op: _Send
2019-06-24 10:57:39.414200: E tensorflow/core/framework/op_kernel.cc:1336] OpKernel ('op: "_Recv" device_type: "GPU"') for unknown op: _Recv
2019-06-24 10:57:39.414371: E tensorflow/core/framework/op_kernel.cc:1336] OpKernel ('op: "_HostSend" device_type: "CPU"') for unknown op: _HostSend
2019-06-24 10:57:39.414661: E tensorflow/core/framework/op_kernel.cc:1336] OpKernel ('op: "_Recv" device_type: "CPU"') for unknown op: _Recv
2019-06-24 10:57:39.414892: E tensorflow/core/framework/op_kernel.cc:1336] OpKernel ('op: "_HostSend" device_type: "GPU" host_memory_arg: "tensor"') for unknown op: _HostSend
2019-06-24 10:57:39.415061: E tensorflow/core/framework/op_kernel.cc:1336] OpKernel ('op: "_HostRecv" device_type: "GPU" host_memory_arg: "tensor"') for unknown op: _HostRecv
2019-06-24 10:57:39.415338: E tensorflow/core/framework/op_kernel.cc:1336] OpKernel ('op: "WrapDatasetVariant" device_type: "CPU"') for unknown op: WrapDatasetVariant
2019-06-24 10:57:39.415613: E tensorflow/core/framework/op_kernel.cc:1336] OpKernel ('op: "WrapDatasetVariant" device_type: "GPU" host_memory_arg: "input_handle" host_memory_arg: "output_handle"') for unknown op: WrapDatasetVariant
2019-06-24 10:57:39.416037: E tensorflow/core/framework/op_kernel.cc:1336] OpKernel ('op: "UnwrapDatasetVariant" device_type: "CPU"') for unknown op: UnwrapDatasetVariant
2019-06-24 10:57:39.416308: E tensorflow/core/framework/op_kernel.cc:1336] OpKernel ('op: "UnwrapDatasetVariant" device_type: "GPU" host_memory_arg: "input_handle" host_memory_arg: "output_handle"') for unknown op: UnwrapDatasetVariant
2019-06-24 10:57:39.416747: I tensorflow/lite/toco/import_tensorflow.cc:1381] Unable to determine output type for op: TFLite_Detection_PostProcess
2019-06-24 10:57:39.439636: F tensorflow/lite/toco/tooling_util.cc:1652] Check failed: input_array_dims[i] == input_array_proto.shape().dims(i) (640 vs. 300)
Fatal Python error: Aborted

Current thread 0x00000238 (most recent call first):
  File "c:\users\alice\anaconda3\lib\site-packages\tensorflow\lite\toco\python\toco_from_protos.py", line 33 in execute
  File "c:\users\alice\anaconda3\lib\site-packages\absl\app.py", line 251 in _run_main
  File "c:\users\alice\anaconda3\lib\site-packages\absl\app.py", line 300 in run
  File "c:\users\alice\anaconda3\lib\site-packages\tensorflow\python\platform\app.py", line 40 in run
  File "c:\users\alice\anaconda3\lib\site-packages\tensorflow\lite\toco\python\toco_from_protos.py", line 59 in main
  File "C:\Users\Alice\Anaconda3\Scripts\toco_from_protos.exe\__main__.py", line 9 in <module>
  File "c:\users\alice\anaconda3\lib\runpy.py", line 85 in _run_code
  File "c:\users\alice\anaconda3\lib\runpy.py", line 193 in _run_module_as_main
AliceDinh commented 5 years ago

@jvishnuvardhan not solved yet

eneshb commented 4 years ago

is here any update on this issue?

wjgaas commented 4 years ago

how to solve this problem? i am still waiting ...