DefTruth
commented
3 years ago Thank you for your great work.
I would like to run the FP16 with CUDA execution provider.
The model rvm_mobilenetv3_fp16.onnx seems to load without problem but when running the inference: auto output_tensors = ort_session->Run(Ort::RunOptions{ nullptr }, input_node_names.data(), input_tensors.data(), num_inputs, output_node_names.data(), num_outputs);
I get an exception.
Do you think that it may be necessary adapt the "create_tensor" helper functions to change the CV_32F opencv mats with CV_16F or make any other change to run the FP16 version?
Best regards.
I think you are right! many thanks to let me know this bug~ can you re-open this issue on lite.ai.toolkit ? i will prefer to fixed this error in lite.ai.toolkit because this demo repo was source from lite.ai.toolkit . You can also provide an implementation of create_tensor_fp16, and then open a new PR, if you prefer~I will review and consider whether to merge your code
Thank you for your great work.
I would like to run the FP16 with CUDA execution provider.
The model rvm_mobilenetv3_fp16.onnx seems to load without problem but when running the inference: auto output_tensors = ort_session->Run(Ort::RunOptions{ nullptr }, input_node_names.data(), input_tensors.data(), num_inputs, output_node_names.data(), num_outputs);
I get an exception.
Do you think that it may be necessary adapt the "create_tensor" helper functions to change the CV_32F opencv mats with CV_16F or make any other change to run the FP16 version?
Best regards.