Problem of quantization with batch_size 2

[WildTaras]

Issue Type

Others

OS

Linux

onnx2tf version number

1.17.5

onnx version number

1.15.0

onnxruntime version number

1.16.3

onnxsim (onnx_simplifier) version number

1.16.3

tensorflow version number

2.15.0

Download URL for ONNX

ref_dataset.zip

Parameter Replacement JSON

Sorry, I dont know

Description

I am sorry for bothering, I had a ticket here https://github.com/PINTO0309/onnx2tf/issues/713 I use this command for full integer quantization. I created reference dataset and attached it. I forgot to mention, that I was using reference dataset with this command onnx2tf -i yolov8n_dynamic_true_batch_size_2.onnx -ois images:2,3,640,640 -cind "images" "ref_dataset.npy" "[[[[0.485,0.456,0.406]]]]" "[[[[0.229,0.224,0.225]]]]" -oiqt -ioqd uint8 As I understand, the shape of the inference data shuold be (2, 640, 640, 3), but when we stack together, I get (imgs_amount*2, 640,640,3), but how onnx2tf should understand that it needs to get tensor (2, 640, 640, 3) for quantization Here is my function for dataset creation:

def create_ref_dataset(dataset, calib_dataset_path, size=640):
    if not os.path.exists(calib_dataset_path):

        img_datas = []
        counter = 0
        for img_path in tqdm(dataset, desc='Processing npy dstaset'):
            if counter > 10:
                break
            img = Image.open(img_path)
            img = img.resize((size, size))
            img = np.array(img)
            img = img[np.newaxis, :]
            img = np.float32(img / 255.0) # probably we need to normalize
            if len(img.shape) != 4:
                continue
            img = np.concatenate([img, img], axis=0)
            print(img.shape)
            img_datas.append(img)
            counter += 1

        calib_datas = np.vstack(img_datas)
        print(f'calib_datas.shape: {calib_datas.shape}')  # [20,640,640,3]
        np.save(file=calib_dataset_path, arr=calib_datas)

[WildTaras]

If I set flag "-b 2" it doesnt help as well. Entire command: onnx2tf -i yolov8n_dynamic_true_batch_size_2.onnx -ois images:2,3,640,640 -cind "images" "ref_dataset.npy" "[[[[0.485,0.456,0.406]]]]" "[[[[0.229,0.224,0.225]]]]" -oiqt -ioqd uint8 -b 2

[PINTO0309]

I finally understand where you're having trouble. It seems that the logic does not take into account the multi-batch quantization pattern. I'll think about it for a bit.

[WildTaras]

Thank you for your patience,

[PINTO0309]

• tests

[PINTO0309]

Fix: https://github.com/PINTO0309/onnx2tf/releases/tag/1.26.2