search

uTensor / utensor_cgen

C++ code generator for uTensor https://utensor-cgen.readthedocs.io/en/latest/
Apache License 2.0
50 stars 40 forks source link

Unit tests for ONNX export/convert #133

Closed victorromeo closed 3 years ago

victorromeo commented 3 years ago

Example of breaking tests

victorromeo commented 3 years ago

Thanks for your feedback. I really appreciate it. To confirm these tests are failing at the moment, as ONNX doesn't contain the shape information required to support the uTensor_cgen operations using the ONNX frontend, so I think these tests will require further work.

dboyliao commented 3 years ago

@victorromeo That's ok. With your tests, I'll take a look this weekend to see if I can fix it. BTW, may I ask why you or your team want to avoid tflite? Just out of curiosity. XD

victorromeo commented 3 years ago

The main reason is to remain as agnostic as possible if the researcher or developer intends to use a framework of choice. The use of ONNX in utensor is a great starting point to achieve this. We have found that different NN frameworks (TensorFlow, PyTorch, MXNet, etc) have resulted in different accuracy outcomes, due to minor implementation eccentricities. As a side benefit, TF appears to undergo more rapid change, and therefore tools such as utensor are more fragile when they rely upon it.

dboyliao commented 3 years ago

I agree. TF has been the main root cause to issues in utensor_cgen for a long time. Not to mention things get worse if you need to trace their python/c++ codes. The code for eager mode is of all evils = =a

dboyliao commented 3 years ago

CI blocks.... I've run the tests manually but it fails. Since it's mostly due to frontend parser, the export api itself should be good. I'll merge it and work on fixing frontend afterward.