[BUG] SuperGradients YOLO-NAS inference not working

Anton-Cherepkov commented 1 month ago

Describe the problem

When using a sample code from the FiftyOne's docs for visualizing the predictions of a super-gradients yolo_nas_m model, the detections do not appear on the FiftyOne's Web UI.

Code to reproduce issue

Provide a reproducible test case that is the bare minimum necessary to generate the problem. Please avoid sharing code that relies on your local data or datasets. Include a short video or screenshot if the bug is in the App.

import fiftyone as fo
import fiftyone.zoo as foz

from super_gradients.training import models

dataset = foz.load_zoo_dataset("quickstart", max_samples=5)
dataset.select_fields().keep_fields()

model = models.get("yolo_nas_m", pretrained_weights="coco")

dataset.apply_model(model, label_field="yolo_nas", confidence_thresh=0.7)

session = fo.launch_app(dataset)

System information

OS Platform and Distribution (e.g., Linux Ubuntu 22.04): Linux Ubuntu 22.04
Python version (python --version): Python 3.10.14
FiftyOne version (fiftyone --version): FiftyOne v0.23.8, Voxel51, Inc.
FiftyOne installed from (pip or source): pip

Other info/logs

The only warning in the console was:

/app/src/super_gradients/training/utils/media/image.py:64: UserWarning: It seems you are using predict() with batch input
  warnings.warn("It seems you are using predict() with batch input")

My investigation showed that:

labels_batch object here holds correct detections

labels object here has type str and does not contain any detections

My conclusion: labels_batch has type Detections, however it seems it should have type List[Detections]

Willingness to contribute

The FiftyOne Community encourages bug fix contributions. Would you or another member of your organization be willing to contribute a fix for this bug to the FiftyOne codebase?

[ ] Yes. I can contribute a fix for this bug independently
[x] Yes. I would be willing to contribute a fix for this bug with guidance from the FiftyOne community
[ ] No. I cannot contribute a bug fix at this time

Anton-Cherepkov commented 1 month ago

Here is a temporary fix for those who may need it: https://github.com/Anton-Cherepkov/fiftyone/commit/aba901bdefb77ed570a3217237d7fd2c42fc35d7

Anton-Cherepkov commented 1 month ago

I noticed that according to the typings this method must return ImagesPredictions object, however it returns the results from another method, which might return not only ImagesPredictions object but also ImagePrediction object.

brimoor commented 1 month ago

@Anton-Cherepkov thanks for catching!

This should be patched by https://github.com/voxel51/fiftyone/pull/4429 and I'm pushing for it to be included in fiftyone==0.24.0 which is scheduled for release this week.

voxel51 / fiftyone