[feature-request] Integrating ultralytics support for Object Detection

[Akash-guna]

Feature type? Integration request (An integration with other libraries and frameworks to increase compatibility)

A proposal draft he proposal is to integrate the Ultralytics library with Nyuntam to expand its functionality and provide users with access to state-of-the-art YOLO models for object detection, segmentation, and classification. This will streamline the process of applying pre-trained models in Nyuntam, enabling faster prototyping and development.

Ultralytics provides ready-to-use models like YOLOv5 and YOLOv8 that are highly optimized and widely supported in various industries, making this integration highly beneficial. Relevant links include the Ultralytics GitHub repository and relevant use cases and demos from the Ultralytics hub. They also provide out of the box quantization.

[joydeep049]

Hello How would we integrate ultralytics with Nyuntam ? Write a separate script which would build the YOLO models supported by ultralytics?

[Akash-guna]

Ultralytics would be integrated into nyuntam vision. You could check how mmdet and mmyolo models are added, and integrate ultralytics similar to that, the guide to adding new model. the important files are the models init file and for an example u can check out timm integration. Basically we load the model in a seperate file named say Ultralytics.py at nyuntam-vision/core/models and add it in the init file and make modifications for custom weight loading. Since ultralytics supports loading custom weights inherently, the existing weight loader can be bypassed.

[joydeep049]

I have been studying the tool and about integrating ultralytics. I have a few questions Am I correct in understanding that: -> Ultralytics integration doesn't involve mmengine or the Runner class? -> Ultralytics would not need a custom register function like we have in timm integration (register_custom_timm_models()) because it primarily uses YOLO ?

@Akash-guna

[joydeep049]

There's an active PR for Ultralytics Integration at https://github.com/nyunAI/nyuntam-vision/pull/20