YOLO_NAS_S and YoloXPostPredictionCallback Fail

[palvors]

🐛 Describe the bug

Hi,

With the model YOLO_NAS_S , the function "non_max_suppression" failed because this part return Tuple and not a integer as expected.

As other exemple , I just want use YoloXPostPredictionCallback

ERROR :

--> 266 candidates_above_thres = prediction[..., 4] > conf_thres # filter by confidence 267 output = [None] * prediction.shape[0] 269 for image_idx, pred in enumerate(prediction):

TypeError: tuple indices must be integers or slices, not tuple

line 266 src: https://github.com/Deci-AI/super-gradients/blob/master/src/super_gradients/training/utils/detection_utils.py

anyone can confirm that ?

Versions

I use super-gradients 3.1.3

see for full requirement of env

asttokens=2.2.1

• backcall=0.2.0
• backports=1.0
• backports.functools_lru_cache=1.6.5
• blas=1.0
• brotlipy=0.7.0
• ca-certificates=2023.7.22
• certifi=2023.7.22
• cffi=1.15.1
• charset-normalizer=2.0.4
• colorama=0.4.6
• cryptography=39.0.1
• cuda-cccl=12.2.53
• cuda-cudart=11.7.99
• cuda-cudart-dev=11.7.99
• cuda-cupti=11.7.101
• cuda-libraries=11.7.1
• cuda-libraries-dev=11.7.1
• cuda-nvrtc=11.7.99
• cuda-nvrtc-dev=11.7.99
• cuda-nvtx=11.7.91
• cuda-runtime=11.7.1
• debugpy=1.5.1
• decorator=5.1.1
• executing=1.2.0
• freetype=2.12.1
• giflib=5.2.1
• idna=3.4
• importlib-metadata=6.8.0
• importlib_metadata=6.8.0
• intel-openmp=2023.1.0
• ipykernel=6.15.0
• ipython=8.14.0
• jedi=0.18.2
• jpeg=9e
• jupyter_client=8.3.0
• jupyter_core=5.3.1
• lerc=3.0
• libcublas=11.10.3.66
• libcublas-dev=11.10.3.66
• libcufft=10.7.2.124
• libcufft-dev=10.7.2.124
• libcurand=10.3.3.53
• libcurand-dev=10.3.3.53
• libcusolver=11.4.0.1
• libcusolver-dev=11.4.0.1
• libcusparse=11.7.4.91
• libcusparse-dev=11.7.4.91
• libdeflate=1.17
• libnpp=11.7.4.75
• libnpp-dev=11.7.4.75
• libnvjpeg=11.8.0.2
• libnvjpeg-dev=11.8.0.2
• libpng=1.6.39
• libsodium=1.0.18
• libtiff=4.5.0
• libuv=1.44.2
• libwebp=1.2.4
• libwebp-base=1.2.4
• lz4-c=1.9.4
• matplotlib-inline=0.1.6
• mkl=2023.1.0
• mkl-service=2.4.0
• mkl_fft=1.3.6
• mkl_random=1.2.2
• nest-asyncio=1.5.6
• openssl=1.1.1l
• packaging=23.1
• parso=0.8.3
• pickleshare=0.7.5
• pillow=9.4.0
• pip=23.1.2
• platformdirs=2.5.2
• prompt-toolkit=3.0.39
• prompt_toolkit=3.0.39
• pure_eval=0.2.2
• pycparser=2.21
• pygments=2.15.1
• pyopenssl=23.0.0
• pysocks=1.7.1
• python=3.9.0
• python-dateutil=2.8.2
• python_abi=3.9
• pytorch=1.13.1
• pytorch-cuda=11.7
• pytorch-mutex=1.0
• pywin32=305
• pyzmq=25.1.0
• requests=2.29.0
• setuptools=67.8.0
• six=1.16.0
• sqlite=3.41.2
• stack_data=0.6.2
• tbb=2021.8.0
• tk=8.6.12
• tornado=6.2
• traitlets=5.9.0
• typing_extensions=4.6.3
• tzdata=2023c
• urllib3=1.26.16
• vc=14.2
• vs2015_runtime=14.27.29016
• wcwidth=0.2.6
• wheel=0.38.4
• win_inet_pton=1.1.0
• xz=5.4.2
• zeromq=4.3.4
• zipp=3.16.2
• zlib=1.2.13
• zstd=1.5.5
• pip:
  • absl-py==1.4.0
  • alabaster==0.7.13
  • antlr4-python3-runtime==4.9.3
  • attrs==23.1.0
  • babel==2.12.1
  • boto3==1.28.9
  • botocore==1.31.9
  • build==0.10.0
  • cachetools==5.3.1
  • click==8.1.6
  • coloredlogs==15.0.1
  • contourpy==1.1.0
  • coverage==5.3.1
  • cycler==0.11.0
  • deprecated==1.2.14
  • docutils==0.17.1
  • einops==0.3.2
  • flatbuffers==23.5.26
  • fonttools==4.41.1
  • future==0.18.3
  • google-auth==2.22.0
  • google-auth-oauthlib==1.0.0
  • grpcio==1.56.2
  • humanfriendly==10.0
  • hydra-core==1.3.2
  • imagesize==1.4.1
  • importlib-resources==6.0.0
  • jinja2==3.1.2
  • jmespath==1.0.1
  • json-tricks==3.16.1
  • jsonschema==4.18.4
  • jsonschema-specifications==2023.7.1
  • kiwisolver==1.4.4
  • markdown==3.4.3
  • markdown-it-py==3.0.0
  • markupsafe==2.1.3
  • matplotlib==3.7.2
  • mdurl==0.1.2
  • mpmath==1.3.0
  • numpy==1.23.0
  • oauthlib==3.2.2
  • omegaconf==2.3.0
  • onnx==1.13.0
  • onnx-simplifier==0.4.33
  • onnxruntime==1.13.1
  • opencv-python==4.8.0.74
  • pip-tools==7.1.0
  • protobuf==3.20.3
  • psutil==5.9.5
  • pyasn1==0.5.0
  • pyasn1-modules==0.3.0
  • pycocotools==2.0.6
  • pydeprecate==0.3.2
  • pyparsing==2.4.5
  • pyproject-hooks==1.0.0
  • pyreadline3==3.4.1
  • pyyaml==6.0.1
  • rapidfuzz==3.1.2
  • referencing==0.30.0
  • requests-oauthlib==1.3.1
  • rich==13.4.2
  • rpds-py==0.9.2
  • rsa==4.9
  • s3transfer==0.6.1
  • scipy==1.11.1
  • snowballstemmer==2.2.0
  • sphinx==4.0.3
  • sphinx-rtd-theme==1.2.2
  • sphinxcontrib-applehelp==1.0.4
  • sphinxcontrib-devhelp==1.0.2
  • sphinxcontrib-htmlhelp==2.0.1
  • sphinxcontrib-jquery==4.1
  • sphinxcontrib-jsmath==1.0.1
  • sphinxcontrib-qthelp==1.0.3
  • sphinxcontrib-serializinghtml==1.1.5
  • stringcase==1.2.0
  • super-gradients==3.1.3
  • sympy==1.12
  • tensorboard==2.13.0
  • tensorboard-data-server==0.7.1
  • termcolor==1.1.0
  • tomli==2.0.1
  • torchaudio==0.13.1
  • torchmetrics==0.8.0
  • torchvision==0.14.1
  • tqdm==4.65.0
  • treelib==1.6.1
  • werkzeug==2.3.6
  • wrapt==1.15.0

[BloodAxe]

It looks like you are using incompatible callback class. For Yolo NAS / PPYolo-E models you should be using PPYoloEPostPredictionCallback(score_threshold=VALUE_YOU_CHOOSE, nms_threshold=VALUE_YOU_CHOOSE, nms_top_k=1000, max_predictions=300) class instead and not YoloXPostPredictionCallback.

Let us know whether it helps.

[palvors]

Hi, Yes but now , I 've a new error with this code

model_nas_s = models.get(Models.YOLO_NAS_S, pretrained_weights="coco")

model_nas_s.eval()

Predict using SG model

with torch.no_grad(): raw_predictions_temp = model_nas_s(transformed_image)

predictions_temp = PPYoloEPostPredictionCallback(score_threshold=0.1,nms_threshold=0.4,nms_top_k=1000, max_predictions=300)(raw_predictions_temp)[0].numpy()

Error -> "TypeError: forward() missing 1 required positional argument: 'device'"

• I added the import : from super_gradients.training.models.detection_models.pp_yolo_e import PPYoloEPostPredictionCallback

sorry, but do you know why ?

[BloodAxe]

Sorry for inconvenience these classes are not quite polished for out-of-trainer usage. You need to pass device= whatever device you placed model to.

Btw, if you want to run inference you can use predict() method that works on numpy images.

[palvors]

Thank you, that work.

so the solution was just to put device= in the foward section like that

predictions_temp = PPYoloEPostPredictionCallback(score_threshold=0.5,nms_threshold=0.6, nms_top_k=1000, max_predictions=300)(raw_predictions_temp,device=None)[0].numpy()

[palvors]

thank you