Live Stream Detection with Specified FPS

[KhonlawatPoon]

❔Question

Hello, I'm an engineering student new to complicated algorithms working with live stream detection. How can I specify fps for video detection? I'm aiming to use it with live stream source from a DJI drone. It seems the processing time doesn't match video FPS. I guess the best way would be skipping some frames to match the video FPS.

Thank you in advance for answers.

Additional context

[github-actions[bot]]

👋 Hello @KhonlawatPoon, thank you for your interest in 🚀 YOLOv5! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset images, training logs, screenshots, and a public link to online W&B logging if available.

For business inquiries or professional support requests please visit https://www.ultralytics.com or email Glenn Jocher at glenn.jocher@ultralytics.com.

Requirements

Python 3.8 or later with all requirements.txt dependencies installed, including torch>=1.7. To install run:

$ pip install -r requirements.txt

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Google Colab and Kaggle notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training (train.py), testing (test.py), inference (detect.py) and export (export.py) on MacOS, Windows, and Ubuntu every 24 hours and on every commit.

[glenn-jocher]

@KhonlawatPoon I don't understand the question. Can you provide a link/documentation or provide more details on your specific use case?

[KhonlawatPoon]

Sorry, I didn't make it clear. @glenn-jocher The question should be video detection, not live stream. I'm testing my trained model with some videos (MP4, 30 fps), and it spends a lot of time. So, I would like to reduce the time by skipping the video frames instead of processing every single frame.

For example, this video contains 430 frames which last around 14 seconds long. _video 9/9 (1/430) C:\Users\User\Documents\GitHub\yolov5\data\images\DJI0093.MP4: 384x640 2 smokes, Done. (2.472s) Which means It takes around 20 minutes to process 14 seconds video. My goal is to reduce processing time by determine the necessary frame rate. (video has 30 fps, process only 10 fps can reduce time by 1/3.)

[glenn-jocher]

@KhonlawatPoon sure, you can customize the image and video dataloader here to skip video frames: https://github.com/ultralytics/yolov5/blob/005d7a8c54a39d89bf2b9dc03fba82a489cd0628/utils/datasets.py#L160-L176

[KhonlawatPoon]

Thank you! It works pretty well now. @glenn-jocher

[SureshbabuAkash1999]

Thank you! It works pretty well now. @glenn-jocher

Should we edit line 174 i.e. self.frame+ =1 to adjust the fps?

[glenn-jocher]

@SureshbabuAkash1999 no this just changes the frame count. See LoadStreams() in datasets.py for an example of skipping frames.

[elgoesto]

Hi @glenn-jocher, I have the same problem but, I can't seem to figure this out. Can you maybe elaborate a bit more on how to skip the frames?

[glenn-jocher]

@elgoesto very simply in https://github.com/ultralytics/yolov5/issues/2621#issuecomment-808734580 every self.cap.read() reads a frame, so you can call it as many times as you want to skip frames.

[kumar665manjeet]

Thank you! It works pretty well now. @glenn-jocher

Hi, I am also trying to skipping frames, but coudn't get it. my modified code.... self.mode = 'video' ret_val, img0 = self.cap.read() if self.frame % skip_frame == 0: ret_val, img0 = self.cap.read() if ret_val: break

   self.frame += 1

I got some changes in output, but need to reduce inference time too reduce. Any help would be appreciated...thanks

[kumar665manjeet]

got it!!!!! thanks

[ZubairKhan001]

Thank you! It works pretty well now. @glenn-jocher

Can you please share the exact changes, i tried using the cap,grab() but it stuck for the last patch of frames that are less than skip frames n = 0 while True: n += 1 self.cap.grab() if n % 30 == 0: # skip frames ret_val, img0 = self.cap.retrieve() if ret_val: break

[glenn-jocher]

@ZubairKhan001 glad to hear you got it! 🎉 If you have any more questions, feel free to ask!