search

Y-T-G / YOLO-NAS-OpenVino-cpp

C++ implementation of YOLO-NAS utilizing OpenVINO backend
MIT License
8 stars 0 forks source link

Hello, can I get some help? #1

Open Ujassi opened 6 months ago

Ujassi commented 6 months ago

First of all, thank you for creating the yolo-nas openvino code. I'm testing your code. I have two questions.

  1. A memory leak occurs. I can't find a way.
  2. If you use ptq using NNCF, detection does not work. Even if you use yolo-nas ptq qat, detection does not occur. NNCF compress_weight works well.

Is there a solution?

Thank you. Have a good day

Y-T-G commented 6 months ago
  1. Is it a big leak? It could he due to OpenCV.
  2. I think that needs some modifications for it to work. I didn't try it.
Ujassi commented 6 months ago

First of all, thank you so much for the answer.

  1. It seems that memory leaks do not occur continuously. There is no problem with testing.

  2. Due to my limited knowledge, I cannot find what needs to be modified to apply the NNCF PTQ. If you have time, I would appreciate it if you could check it out later.

Thank you again for your reply.

Y-T-G commented 6 months ago

  1. It could be OpenCV related: https://github.com/opencv/opencv/issues/5715?notification_referrer_id=MDE4Ok5vdGlmaWNhdGlvblRocmVhZDExMTYwNzQzOTozMjIwNjUxMQ%3D%3D#issuecomment-2067695016

  2. Can you try adding these after the line https://github.com/Y-T-G/YOLO-NAS-OpenVino-cpp/blob/2856838d9004b0c097d3577f904db5197944325b/src/yolo_nas.cpp#L51

    ppp.output(0).tensor().set_element_type(ov::element::f32);
ppp.output(1).tensor().set_element_type(ov::element::f32);
Ujassi commented 6 months ago

Thank you for your answer.

  1. I don't use VideoCapture, I use images.
  2. Even if you enter the code you mentioned, detection does not occur. 1) If xxx.bin xxx.xml is created with f32, it works well. 2) If xxx.bin xxx.xml is created with f16, it works well. 3) Make xxx.bin xxx.xml as f32, Following https://github.com/openvinotoolkit/nncf/tree/develop?tab=readme-ov-file, we conducted PTQ with the code below. However, detection does not occur.

import nncf import openvino.runtime as ov import torch from torchvision import datasets, transforms

model = ov.Core().read_model("/model_path")

val_dataset = datasets.ImageFolder("/path", transform=transforms.Compose([transforms.ToTensor()])) dataset_loader = torch.utils.data.DataLoader(val_dataset, batch_size=1)

def transform_fn(dataitem): images, = data_item return images

calibration_dataset = nncf.Dataset(dataset_loader, transform_fn)

quantized_model = nncf.quantize(model, calibration_dataset)

ov.save_model(quantized_model, "/model_path")

4) PTQ and QAT were conducted following https://github.com/naseemap47/YOLO-NAS. But it doesn't work.

Could I possibly get some help?

Thank you so much.

Y-T-G commented 6 months ago
  1. How does it leak for images? Doesn't it stop after each image?
  2. You can try checking the output casting. I think it has something to do with going from int to float.
Ujassi commented 6 months ago
  1. It seems to occur only once when creating an object of the YoloNAS class. I will check this part a little more.
  2. I am thinking similarly, but I still can't find a way.

Thank you so much for your reply.