How to add preprocessing steps for quantized yolov8n (float 16 or int8) in onnx format?

[feff2]

I tried to add pre- and post- processing steps to yolov8n. model = onnx.load('models/detect/yolov8n_pricetag_segment_fp16.onnx') inputs = [create_named_value('image', onnx.TensorProto.UINT8, ['num_bytes'])]

pipeline = PrePostProcessor(inputs, 17)

pre_processing_steps = [ ConvertImageToBGR(), Resize((640, 640), policy='not_larger'), LetterBox(target_shape=(640, 640)), ChannelsLastToChannelsFirst(), ImageBytesToFloat(), Unsqueeze([0]), ]

post_processing_steps = [ Squeeze([0]), Transpose([1, 0]), Split(num_outputs=2, axis=-1, splits=[4, 1]),
SelectBestBoundingBoxesByNMS( iou_threshold=0.5, score_threshold=0.5), (ScaleNMSBoundingBoxesAndKeyPoints(name='ScaleBoundingBoxes'), [ utils.IoMapEntry('ConvertImageToBGR', producer_idx=0, consumer_idx=1), utils.IoMapEntry('Resize', producer_idx=0, consumer_idx=2), utils.IoMapEntry('LetterBox', producer_idx=0, consumer_idx=3), ]) ]

pipeline.add_pre_processing(pre_processing_steps) pipeline.add_post_processing(post_processing_steps)

but when I try to start an ort session

detect_session = onnxrt.InferenceSession( 'models/detect/yolov8n_pricetag_segment_fp16_modified.onnx', sess_options=session_options, providers=providers )

I get the following error

onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Load model from models/detect/yolov8n_pricetag_segment_fp16_modified.onnx failed:Type Error: Type parameter (T) of Optype (Conv) bound to different types (tensor(float) and tensor(float16) in node (/model.0/conv/Conv).

[feff2]

if we look at the graph of the model, then before the first convolution we will see that the data in float 32 but in convolution layer data in float 16

[feff2]

this is outputs