Simultaneous Use of a Model by Multiple Users for Tracking

SIG777 commented 2 weeks ago

Search before asking

[X] I have searched the Ultralytics YOLO issues and discussions and found no similar questions.

Question

The server uses an object detection model for the tracking task. For optimization purposes, it is loaded into GPU memory and then used. I have multiple users who will be working with the same loaded model. How can I ensure that their work will not be mixed with each other? I would not want to implement my own identification and tracking algorithm on top of the detection. I would also not want to load the same model into memory multiple times depending on the number of user sessions. What is the best practice for tracking a detection model simultaneously by multiple different users? I look forward to your expert opinion. Thank you.

Additional

No response

UltralyticsAssistant commented 2 weeks ago

👋 Hello @SIG777, thank you for reaching out to the Ultralytics community 🚀! This is an automated response, and an Ultralytics engineer will assist you soon.

For your query about utilizing a model for multiple users without mixing results, we suggest checking our Docs for guidance. You might find useful approaches on handling concurrent requests and model usages there.

As you're tackling a custom scenario, please share further specifics like environment details or implemented methods. If it's a 🐛 Bug Report, a minimum reproducible example would be greatly appreciated to help us assist you better.

For real-time support, feel free to join our Discord community 🎧, or engage in discussions on Discourse. Our Subreddit is also a great space to share insights.

Upgrade

Make sure you're using the latest version of the ultralytics package with all requirements installed in a Python>=3.8 environment with PyTorch>=1.8. You can upgrade using:

pip install -U ultralytics

Environments

For the best experience, consider running YOLO in any of these verified environments with dependencies preinstalled:

Notebooks with free GPU:
Google Cloud VM: See our GCP Quickstart Guide
Amazon AMI: See our AWS Quickstart Guide
Docker Image: Check the Docker Quickstart Guide

Status

Keep an eye on the CI badge above; if green, all our Ultralytics CI tests are passing, ensuring reliability across different environments.

We look forward to assisting you further!

Y-T-G commented 2 weeks ago

With ultralytics, the easiest way to achieve this would be to use Triton Inference Server to serve the model:

https://docs.ultralytics.com/guides/triton-inference-server/

That way the model is decoupled from the client, which means you can create multiple model instances using ultralytics without loading the actual model as all of them will be using the model in the Triton Inference Server.

So you can create a model instance for each user and use the model.track as you normally do. Each user's session will be separate, so they won't interfere.

SIG777 commented 2 weeks ago

With ultralytics, the easiest way to achieve this would be to use Triton Inference Server to serve the model:

https://docs.ultralytics.com/guides/triton-inference-server/

That way the model is decoupled from the client, which means you can create multiple model instances using ultralytics without loading the actual model as all of them will be using the model in the Triton Inference Server.

So you can create a model instance for each user and use the model.track as you normally do. Each user's session will be separate, so they won't interfere.

@Y-T-G, thanks for the answer. I'll keep that in mind.

ultralytics / ultralytics