Open ogoun opened 1 year ago
I also got the same issue. Any updates on this?
Based on my research, there is no documented way to convert an ML.NET model for object detection into the ONNX format. The issue seems to be the MLImage object. The reason for this appears to be that the ONNX Estimator expects the input to be in certain formats, and MLImage isn't one of them (see also the reply from @LittleLittleCloud here to an simlar question).
After looking into ONNX, I had an idea of how it could work. Unfortunately, I lack some knowledge about how the training process of ModelBuilder works, but I have a hunch that the trained model's architecture in the saved ML.NET file contains the model input. From the documentation, it seems that simply converting an ML.NET model that contains an MLImage to the ONNX format would be impossible. Therefore, I think the ModelInput needs to be changed before training the model.
If retraining your model is an option:
My approach would be to modify the code generated by ModelBuilder so that our model input contains only data types that the ONNX Estimator can handle. We would then retrain our ML.NET model and save it in the ONNX format.
The steps to do this would be:
Modifying our ModelInput/ModelOutput
Convert the MLImage into a data type that the ONNX Estimator can handle. Perhaps we could change/convert the image/model input and output to something like:
public float[] ImageAsTensor { get; set; }
// Additionally, if we want to avoid changing the logic of some functions in the generated code, it would be wise to save the image dimensions.
public float ImageWidth { get; set; }
public float ImageHeight { get; set; }
Writing additional helper functions
Create a function like ConvertMLImageToTensor
.
Extensive modification of the code generated by ModelBuilder
Modify the code to handle the datatype float[] ImageAsTensor
instead of MLImage Image
.
Updating the save method
Update the save method to use Microsoft.ML.Model.OnnxConverter
to save an ONNX file.
Calling the Train function
I haven't tried this, so there is no guarantee that it will work.
Alternatively, instead of making so many modifications to your code and training the model locally on the CPU/GPU, it would likely be much more time-efficient to simply select Azure in the environments. This option directly outputs a model in the ONNX format.
I am having the same problem, I think ML should consider being more ONNX export friendly on the template builders.
So, for example, before training step, to have an option like this:
☑ do all neccesary changes to ensure the trained model is exportable to ONNX
So if that option is checked, the input columns are modified to be Onnx friendly, as @MasterNaz has suggested.
And afterwards, if the trained model is Onnx friendly, then add another option to include the code to export to ONNX
Another option would be, given that MLImage is specific type with a well known conversion to a tensor, to reinterpret Microsoft.ML.Data.MLImage as a tensor when creating the ONNX model instead of throwing an exception
System Information:
Describe the bug Fault conversion to ONNX
To Reproduce Steps to reproduce the behavior:
Try convert model
public static void Export() { var onnxPath = "test.onnx"; var mlContext = new MLContext(); ITransformer mlModel = mlContext.Model.Load(MLNetModelPath, out var _);