About the problem that after quantizing the model, the model cannot reason and the accuracy is greatly reduced.

[chenjiafu-George]

Hi @wondervictor ,I changed the associated config, checkpoint, and img-size in export_onnx.py. The program worked fine and generated the onnx model, but after I ran onnx_demo.py, the inferential images didn't work as expected. Result of running onnx_demo.py with the quantized model: Result of running onnx_demo.py with the pre-quantization model:

My initial suspicion is that I changed the input size of the model from 640x640->1280x384.But there is no good solution at present. Do you have any suggestions?Your reply will be greatly appreciated!

[wondervictor]

Hi @chenjiafu-George, you need to check:

1. the image resolution used during training. If you want to infer using 1280x384, you'd better load the pre-trained models and fine-tune YOLO-World with 1280x384 resolution,
2. the postprocessing, it seems that the results have offsets.

[wondervictor]

BTW, you can export the 640x640 first and evaluate the performance. And this guide/demo might be helpful.

[chenjiafu-George]

你好@chenjiafu-George，您需要检查：

1. 训练期间使用的图像分辨率。如果你想使用 进行推断1280x384，你最好加载预训练的模型并微调 YOLO-World 的1280x384分辨率，
2. 后处理，看起来结果有偏移。

Hi @wondervictor ,I did modify the input_size 1280x384 in the training of the problem you said, and the trained model has achieved good results. I am satisfied with it, but the exported model does not work well now. I also tried to use the 640x640 model for export work. The 640x640 model can run well in the version of export_v1. At present, I seem to have some problems with your newly updated export.py: 1.The exportable.py code works, but the exported model is only 1kb in size.No error was reported during the operation. I used the netron tool to visualize the model, as shown below: My inference is that the quantification seems to have failed, right? 2.onnx_demo.py runs with an error.

[chenjiafu-George]

你好@chenjiafu-George，您需要检查：

1. 训练期间使用的图像分辨率。如果你想使用 进行推断1280x384，你最好加载预训练的模型并微调 YOLO-World 的1280x384分辨率，
2. 后处理，看起来结果有偏移。

Hi @wondervictor ,for your reply 2, aboat the postprocessing, it seems that the results have offsets. This may be a problem in the quality of my data set labeling. I will carry out data governance and correct the labeling box in the future.I'm not worried about that at the moment. It's mainly my quantitative model that's going wrong.

[wondervictor]

Hi @chenjiafu-George, you need to specify the --custom-texts during exporting. If you want to quantize the model with TFLite, you need an extra --without-bbox-decoder. You can refer to: https://github.com/AILab-CVC/YOLO-World/blob/master/docs/tflite_deploy.md

[chenjiafu-George]

你好@chenjiafu-George，您需要--custom-texts在导出过程中指定。如果您想使用 TFLite 量化模型，则需要额外的--without-bbox-decoder.您可以参考：https://github.com/AILab-CVC/YOLO-World/blob/master/docs/tflite_deploy.md

Hi @wondervictor, Sorry to bother you so late. Yes, I do use such argument, but I have only exported model 1 KB, I tried many times to other PTH model are derived, and the result is 1 KB. I use the CMD command is: python deploy/export_onnx_v2.py E:\george\YOLO-World\configs/pretrain/yolo_world_v2_s_vlpan_bn_2e3_100e_4x8gpus_obj365v1_goldg_train_1280ft_lvis_minival.py E:\george\YOLO-World\tools/work_dirs\yolo_world_v2_s_vlpan_bn_2e3_100e_4x8gpus_obj365v1_goldg_train_1280ft_lvis_minival_KITTI——Adamw-0.9/best_coco_bbox_mAP_epoch_99.pth --opset 12 --without-bbox-decoder custom-text is not specified because the default value is set in the program. classes.json:

[wondervictor]

Have you used the latest code, including easydeploy?

[chenjiafu-George]

Have you used the latest code, including easydeploy?

Hi @wondervictor I'm using the latest version of easydeploy. After use, it is indeed 1kb. I tried to switch back to the previous version, and the exported model is not 1kb.Can provide about your latest export onnx of logs?Thank you so much.

[chenjiafu-George]

Have you used the latest code, including easydeploy?

Hi @wondervictor ,Is there a problem with the Clip version? When I ran the previous version, this message didn't appear. I don't know exactly why.

[wondervictor]

Hi @chenjiafu-George, it's normal since we do not use the vision model of the pre-trained CLIP.

[chenjiafu-George]

Hi @wondervictor ,I'm sorry to disturb you. I pulled the project code again and tried again. Now the problem has been solved, and the problem may be caused by the conflict of code when I was pulling in git. Next I tried quantizing the model using int8.Thank you for your patience!

[wondervictor]

Have you quantized YOLO-World with TFLite? You can try the tflite demo here. Notably, TFLite does not support GPU which is more suitable for mobile/edge devices.

[chenjiafu-George]

Have you quantized YOLO-World with TFLite? You can try the tflite demo here. Notably, TFLite does not support GPU which is more suitable for mobile/edge devices.

Hi @wondervictor ,Yes, I am currently trying to quantify the int8 model. After I try it, I will transplant it to the jetson nano board for testing and let you know the test results as soon as possible.

[wondervictor]

It might be better to use TensorRT to deploy YOLO-World on Jetson Nano. You can try to use "ONNX to TensorRT", such as https://github.com/onnx/onnx-tensorrt. We appreciate it much if you could evaluate the ONNX models with TensorRT, it's really helpful!

[chenjiafu-George]

Have you quantized YOLO-World with TFLite? You can try the tflite demo here. Notably, TFLite does not support GPU which is more suitable for mobile/edge devices.

Hi @wondervictor ,What version of onnx2tf are you using?

[chenjiafu-George]

It might be better to use TensorRT to deploy YOLO-World on Jetson Nano. You can try to use "ONNX to TensorRT", such as https://github.com/onnx/onnx-tensorrt. We appreciate it much if you could evaluate the ONNX models with TensorRT, it's really helpful!

Hi @wondervictor ,Ok, I'll do my best to finish the job.Thank you again for your support！In addition, I will also do the work of int8 quantification. In short, thank you very much for your support to my work. Your open source code is very helpful to me.

[wondervictor]

Yep, the next target of our plan is to release TensorRT with INT8 Quantization, and your work will be helpful!

[chenjiafu-George]

你用 TFLite 量化过 YOLO-World 吗？您可以在这里尝试tflite 演示。值得注意的是，TFLite 不支持 GPU，更适合移动/边缘设备。

Hi @wondervictor ,In the process of quantifying the int8 model, I used cmd: onnx2tf -i E:\george\YOLO-World\deploy\work_dir\yolo_world_v2_s_COCO\yolo_world_v2_s_vlpan_bn_2e-4_80e_8gpus_mask-refine_finetune_coco_ep80-492dc329.onnx -oiqt -cind "images" "E:\george\YOLO-World\deploy\tflite_calibration_data_100_images_640.npy" "[[[[0.,0.,0.]]]]" "[[[[1.,1.,1.]]]]" -onimc "scores" "bboxes" --verbosity debug Throwing an exception:

Have you encountered this problem?

[wondervictor]

How about the shape of your calibration data? The shape of the calibration data should be consistent with the ONNX input.

[chenjiafu-George]

In addition, I have a question, yolov8 can directly call yolo-worldv2s.pt, about the size of this yolo-worldx2s.pt file is not large, may I ask how you made it, maybe this model can also meet my needs.

[chenjiafu-George]

How about the shape of your calibration data? The shape of the calibration data should be consistent with the ONNX input.

Hi @wondervictor ,The current size of the npy data file is (100, 640, 640, 3), I used netron to look at the network structure: I will now change the data file to (1,640,640,3) and try again to see if I can quantify success.

[wondervictor]

The calibration data is ok.

[wondervictor]

It seems the numpy data file is corrupted and you can re-generate it.

[chenjiafu-George]

Yep!Thank you for your reply, which helped me solve this problem. it works now, but there's another issue: it seems to be my tensorflow version that's causing some syntax incompatibility, right now My tensorflow version is 2.13.0, My python version is 3.8, The highest version that pip will find is 2.13.1, so it seems that there is no way to install higher versions of tensorflow using pip. The latest tensorflow version was not found:

[wondervictor]

You should update PyTorch to >=3.9 to install tensorflow==2.16.1, which is required to use onnx2tf.

[chenjiafu-George]

You should update PyTorch to >=3.9 to install tensorflow==2.16.1, which is required to use onnx2tf.

OK,I'm using Anaconda to switch to a higher python version and install tensorflow==2.16.1, which means Python 3.8 supports maximum tensorflow version 2.13.1.

[chenjiafu-George]

Hi @wondervictor ,I changed python version to 3.10, tensorflow version 2.16.1, and now using onnx2tf to export tflite model has been successful!But there seems to be an error in tflite_demo.py: (1).the argument path in parser.add_argument('path', help='TFLite Model '.tflite ') should be changed to tflite. (2).This is not an error, just a directory adjustment, parameter passing problem. (3).About IndexError: list index out of range I output all the information of output_details, and it is true that there is no index 1. I don't know why here. Then I changed interp.get_tensor(output_details[1]['index']) to interp.get_tensor(output_details[0]['index']). get the following inference graph:

[wondervictor]

Hi, how about the inference scripts? The output_index may change due to different models and it's hard to determine.

[chenjiafu-George]

Hi, how about the inference scripts? The output_index may change due to different models and it's hard to determine.

Hi @wondervictor ，I've used tflite_demo.py for inference.

[chenjiafu-George]

Hi @wondervictor ，In addition, thank you very much for your answer during this period of time. I look forward to your release of the quantization model on tensorRT version！I will continue to pay attention to it. Today, I discussed with the professor about the proposal to publish a paper in IEEE Transactions on Intelligent Transportation Systems(TITS). If you approve this journal, I sincerely hope to cooperate with you on a paper. I am still working on the experiment part of the paper.

[wondervictor]

Hi @chenjiafu-George, nice work! We're honored to hear that you want to use YOLO-World in your academic paper. If there are subsequent problems or points you think can be enhanced, we welcome discussions.

[chenjiafu-George]

Hi @wondervictor ，I would like to ask about the weight file of yolov8s-worldv2.pt you published on the official website of ultralytics, which is only 24.7MB. May I ask how you did it? It would be helpful if you could provide me with relevant code.

[chenjiafu-George]

Hi @wondervictor , Sorry to bother you, but is there any progress on inference acceleration for TensorRT recently?

[wondervictor]

Hi @chenjiafu-George, our progress has stalled, but we will still organize everything as soon as possible.

[chenjiafu-George]

Hi @wondervictor , I recently learned that an article about the quantitative VIT, whole lot address: https://github.com/megvii-research/FQ-ViT, you see whether with a little help to you.if you see to your address: https://arxiv.org/pdf/2111.13824.pdf.

[chenjiafu-George]

Hi @wondervictor , Code for quantifying models and evaluating them,Has it been updated recently?