Question about DL model deployment for autonomous driving usage

TexasInstruments / edgeai

Edge AI Software and Development Tools

BSD 3-Clause "New" or "Revised" License

124 stars 13 forks source link

Question about DL model deployment for autonomous driving usage #3

Closed zengpeng629 closed 2 years ago

zengpeng629 commented 2 years ago

Hi TI team,

I am currently working on deploying a detection model on tda4 and aiming to make it work for AD usage in the future (for cars). I followed edgeai tutorials and documents and successfully ran inference on TDA4VM with ONNX Runtime Interface using python, so here is my question:

For autonomous driving, is it possible to use onnx runtime on tda4 with python for real industrial usage? Since the development process with ONNXRT is much more easier compared to use TIOVX (pure C), but I am not sure about the efficiency or safety. Could you share some information about if there are companies are making their perception system based on open source runtime with TIDL on tda4? Or those runtimes with TIDL is just for experiments, for industrial usage, maybe only TIOVX is acceptable?

Best, Zeng

mathmanu commented 2 years ago

You might have noticed that it is possible to use ONNXRT with C++ as well and there are examples for it (you are not restricted to Python). Those high level opensource runtimes are not just for experiments - they are for deployment as well. Efficiency is very good - as long as the graph is fully offloaded to underlying TIDL there is not much efficiency loss.

Your question whether to use the high level gstreamer interface in StarterKit or the low level TIOVX interface. I think every company will have their own strategy regarding the tools to use - it's hard to generalize. If your company uses Linux then I think it is a good idea to use the high level runtimes. But if you only run RTOS in the SoC then your only option is to use TIOVX.

zengpeng629 commented 2 years ago

Hi Manu,

Thank you again for your kind help!

It is more clear now about the deployment process! Another question is that could you share some document links where I can learn about how the graph offloading works?

mathmanu commented 2 years ago

Hi Zeng,

Have you looked at the TIDL documentation? For the 8.0 release, it is here: https://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-jacinto7/08_00_00_12/exports/docs/tidl_j7_08_00_00_10/ti_dl/docs/user_guide_html/index.html This should give you some details.

The exact offloading mechanism is specific to the runtime. For example, see these links: https://onnxruntime.ai/docs/execution-providers/ https://www.tensorflow.org/lite/performance/delegates

zengpeng629 commented 2 years ago

Thank you so much!

I will check the documents, those are very helpful to me!

Best regards, Zeng