stream＝True？

[HornGate]

Search before asking

• [X] I have searched the YOLOv8 issues and discussions and found no similar questions.

Question

When I use yolov8x.pt (for detect) to predict the number of images in the folder greater than 1000, it show me with warnings that I should make sure the stream=True, otherwise it would easily result in OOM. How should I set stream = True?

Additional

No response

[Laughing-q]

@HornGate just pass stream=True and use a for loop to launch it.

results = model(source=..., stream=True)  # generator of Results objects
for r in results:
    boxes = r.boxes  # Boxes object for bbox outputs
    masks = r.masks  # Masks object for segment masks outputs
    probs = r.probs  # Class probabilities for classification outputs

[dh031200]

You can use stream=True like this: model = YOLO("yolov8n.pt")

results = model.predict(source=src, verbose=False, stream=True)
for r in results:
    writer.write(r.plot())

[glenn-jocher]

Thanks for sharing your solution! Indeed, setting stream=True to enable for-loop processing can prevent Out of Memory (OOM) issues due to reducing intermediate tensor storage. It's great to see that the new predict API of YOLOv8 can handle online video processing with stream=True.

[apiszcz]

Using the yolov8 CLI how does one enable stream=True?

[glenn-jocher]

@apiszcz stream=True is enabled by default for CLI inference, but if you have any doubt you can force it with yolo predict stream=True

[HornGate]

yolo predict stream=True

But why we get this "SyntaxError: 'stream' is not a valid YOLO argument. "?

[glenn-jocher]

@HornGate i apologize for the confusion. The stream argument is actually not a CLI argument of YOLOv8. It's a parameter you pass to the predict method when using the YOLOv8 Python API.

So to clarify, you don't need to enable stream=True when using yolo predict CLI command. The CLI command automatically enables stream=True mode to process videos or live streams in real-time.

I hope this clears up any confusion. Let me know if you have any other questions!

[HornGate]

@HornGate i apologize for the confusion. The stream argument is actually not a CLI argument of YOLOv8. It's a parameter you pass to the predict method when using the YOLOv8 Python API.

So to clarify, you don't need to enable stream=True when using yolo predict CLI command. The CLI command automatically enables stream=True mode to process videos or live streams in real-time.

I hope this clears up any confusion. Let me know if you have any other questions!

Thank you.

[apiszcz]

yolov8 started with CLI is producing this warning

WARNING  stream/video/webcam/dir predict source will accumulate results in RAM unless `stream=True` is passed,
    causing potential out-of-memory errors for large sources or long-running streams/videos.

[glenn-jocher]

@HornGate That warning is simply to inform you that if you don't pass stream=True to the predict method or to the yolo CLI command, YOLOv8 will store all the detected results in RAM.

This can be a problem for large videos or long-running processes and can lead to Out of Memory (OOM) errors. So to avoid those issues, it's recommended to enable stream=True when processing larger sources with YOLOv8.

If you're not processing a particularly large source with YOLOv8, you can safely ignore the warning.

[HornGate]

yolov8 started with CLI is producing this warning

WARNING  stream/video/webcam/dir predict source will accumulate results in RAM unless `stream=True` is passed,
    causing potential out-of-memory errors for large sources or long-running streams/videos.

You can run “yolo predict cfg=./predict_cfg.yaml”，and this CLI command automatically enables "stream=True" mode to process videos or live streams in real-time.

[apiszcz]

That is what I am doing, I am still getting the warning.

Tx

From: HornGate @.> Sent: Friday, March 31, 2023 12:27 PM To: ultralytics/ultralytics @.> Cc: ap @.>; Mention @.> Subject: Re: [ultralytics/ultralytics] stream＝True？ (Issue #1713)

yolov8 started with CLI is producing this warning

WARNING stream/video/webcam/dir predict source will accumulate results in RAM unless stream=True is passed, causing potential out-of-memory errors for large sources or long-running streams/videos.

You can run “yolo predict cfg=./predict_cfg.yaml”，and this CLI command automatically enables "stream=True" mode to process videos or live streams in real-time.

— Reply to this email directly, view it on GitHub https://github.com/ultralytics/ultralytics/issues/1713#issuecomment-1492234809 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AAK5KTM47SCHF47JXIN2CH3W64AVZANCNFSM6AAAAAAWM4NETU . You are receiving this because you were mentioned. https://github.com/notifications/beacon/AAK5KTLFNIV4WUIMCSHGW2TW64AVZA5CNFSM6AAAAAAWM4NETWWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTSY6GZDS.gif Message ID: @. @.> >

[glenn-jocher]

@HornGate The warning message is there to inform you of the potential issue of running out of memory. If your source files are not too large, or if you are processing them relatively quickly, you can ignore the message.

If you need to process larger files, or if your processing is particularly long-running, then you should pass the stream=True argument to the yolo predict command to avoid accumulating results in RAM.

If you have already passed stream=True and are still getting the warning messages, you can safely ignore them as long as you are able to process your files without any issues or crashes.

I hope this helps clarify your understanding. Let me know if you have any other questions!

[apiszcz]

stream=True throws an error stating it is invalid option on the command line or in a cfg file

yolo cfg=test.yml stream=True

or

yolo cfg=test.yaml where test.yaml has stream: True

[glenn-jocher]

@HornGate I apologize for the confusion. The stream parameter is not a recognized parameter in the YOLOv8 configuration file or CLI command.

However, you can configure the streaming mode by setting the source parameter to a URL, camera index, or directory path to automatically turn on streaming mode. For example, source: "rtsp://192.168.1.10:554/stream1" to stream from a network camera, or source: 0 to stream from the default local camera.

If you are processing files from a directory, make sure to include the directory path in the source parameter, such as source: "/path/to/directory/".

I hope this helps. Let me know if you have any other questions.

[apiszcz]

I am. Processing is running fine, curious about the warning. Got it so it is for streaming sources, file lots of video files on disk Again thanks for adding --vid-stride Only other feature that I would like to see is recursive folder search for files.

[glenn-jocher]

@HornGate You're welcome! I'm glad to hear that the processing is running fine.

Yes, the warning is mostly for streaming sources, but it's always a good idea to keep your system resources in mind when processing large amounts of data.

Regarding recursive folder search, it's a good suggestion, and I will pass it along to the development team for consideration. In the meantime, as a workaround, you can use the command-line tools provided by your operating system to create a list of files across multiple directories and pass that list to YOLOv8 using the source option.

Let me know if you have any other questions or concerns.

[apiszcz]

thanks, was not aware of list input for source, i have a lot of files.

[glenn-jocher]

@HornGate You're welcome! Yes, the source option in YOLOv8 allows you to input a list of files for processing, making it easy to process multiple files at once without the need to include individual file paths in the command.

To use this option, simply create a text file listing the file paths (one file path per line), and pass the file path to that text file as the source. The source option supports text files with the .txt extension.

I hope this helps simplify your data processing workflow. Let me know if you have any other questions or concerns.

[HornGate]

stream=True throws an error stating it is invalid option on the command line or in a cfg file

yolo cfg=test.yml stream=True

or

yolo cfg=test.yaml where test.yaml has stream: True

@apiszcz You should run "yolo predict cfg=...."

[apiszcz]

Did i miss the the --source listofiles.txt in the documentation?

[apiszcz]

yolov8 running CLI on a lot of video files, memory use > 210GB? Instance 1> 200GB Instance 2> 120GB

Seems like there is an issue.

[apiszcz]

Switching back to yolov5. Standing by for any tips on how to conserve memory. yv5 stays under 2.8 GB RES RAM per process.

[glenn-jocher]

@AyushExel @Laughing-q seems like YOLOv8 RAM usage during video/dir CLI inference is much higher than YOLOv5 per @apiszcz above

[apiszcz]

After a few hours of operation yv8 exceeds 600GB of RAM and stalls the system. yv5 no issues, stays at 2.8GB through processing runs. I have tried yv8 on a few different instances, ubuntu 20.04. I was hoping to compare detection performance between yv5 7.0 and yv8.

[AyushExel]

This points to memory leak. I'll try to reproduce tonight

[Laughing-q]

@apiszcz could you provide your command to reproduce?

[AyushExel]

@apiszcz cli mode already uses stream mode by default so you should not get that warning. We can try to debug it if you provide the reproducible command. Thanks!

[apiszcz]

yolo cfg=yolov8cfg.yml

From: Ayush Chaurasia @.> Sent: Sunday, April 2, 2023 1:55 AM To: ultralytics/ultralytics @.> Cc: ap @.>; Mention @.> Subject: Re: [ultralytics/ultralytics] stream＝True？ (Issue #1713)

@apiszcz https://github.com/apiszcz cli mode already uses stream mode by default so you should not get that warning. We can try to debug it if you provide the reproducible command. Thanks!

— Reply to this email directly, view it on GitHub https://github.com/ultralytics/ultralytics/issues/1713#issuecomment-1493231660 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AAK5KTI2PHZLWNCMHBSBIFDW7EICLANCNFSM6AAAAAAWM4NETU . You are receiving this because you were mentioned. https://github.com/notifications/beacon/AAK5KTOJGXESE3XQ3JEAN6DW7EICLA5CNFSM6AAAAAAWM4NETWWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTSZADUCY.gif Message ID: @. @.> >

[apiszcz]

rcvd an email stating this issue is closed?

[AyushExel]

@apiszcz I'm reopening this issue. I think we know why this is happening. Somehow when you pass cfg=... the stream mode is not being initialized. I'll investigate this after the current project. Please ping me if you don't hear from me in 2 days

[apiszcz]

I agree, which is the issue raised in the start of this issue. Thank you!

---

From: Ayush Chaurasia @.> Sent: Monday, April 3, 2023 10:12 AM To: ultralytics/ultralytics @.> Cc: ap @.>; Mention @.> Subject: Re: [ultralytics/ultralytics] stream＝True？ (Issue #1713)

@apiszczhttps://github.com/apiszcz I'm reopening this issue. I think we know why this is happening. Somehow when you pass cfg=... the stream mode is not being initialized. I'll investigate this after the current project. Please ping me if you don't hear from me in 2 days

— Reply to this email directly, view it on GitHubhttps://github.com/ultralytics/ultralytics/issues/1713#issuecomment-1494402082, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAK5KTLGALC5H47EFY42KULW7LLFNANCNFSM6AAAAAAWM4NETU. You are receiving this because you were mentioned.Message ID: @.***>

[glenn-jocher]

Apologies for the confusion. Yes, it seems like this issue still remains unsolved. When passing cfg=... it appears that the stream mode is not being initialized which could be causing the issue. @apiszcz has provided a command earlier for reproducing the issue, and I will investigate this after finishing my current project. If you don't hear from me in 2 days or need further assistance, please don't hesitate to ping me again.

[Laughing-q]

@apiszcz @AyushExel @glenn-jocher well I'm able to reproduce the issue with yolo cfg=test.yaml which the source in this file is a video, but in fact the author @HornGate of this issue had commented the correct usage twice. https://github.com/ultralytics/ultralytics/issues/1713#issuecomment-1492697658 and https://github.com/ultralytics/ultralytics/issues/1713#issuecomment-1492234809 you should use yolo predict cfg= instead of yolo cfg=. This is because the updated logic here: https://github.com/ultralytics/ultralytics/blob/4198570a4ba98351107527142d85125eb8d0039f/ultralytics/yolo/engine/model.py#L223-L224

EDIT: closing the issue since yolo predict works as expect and I don't see any unusual memory increasing in my tests. :)

[apiszcz]

Thank you will test.

[glenn-jocher]

@apiszcz you're welcome! The solution to your problem might be as simple as using yolo predict cfg= instead of yolo cfg=. This is because the yolo cfg= command doesn't initialize the stream mode properly when the cfg file contains a video source, which could be causing the out-of-memory issue. The correct command yolo predict cfg= should initialize the stream mode properly and avoid the out-of-memory issue.

[apiszcz]

5.2gb vs 100GB+ per process. Thanks. still waiting for the REST API

[glenn-jocher]

@apiszcz i'm glad the solution worked for you! As for the REST API, it should be a great addition to Ultralytics YOLOv8. REST API is a way to provide an HTTP interface for other systems to interact with your application. It will allow you to serve your YOLOv8 model as an HTTP service in order to perform real-time object detection of live video streams, image files, and IP camera feeds. This can be useful for a number of applications, such as security cameras, traffic monitoring, and more.

[apiszcz]

Is the REST API ready? If so, i missed the folder. Thanks.

[glenn-jocher]

@apiszcz, as far as I know, the REST API is not yet available in Ultralytics YOLOv8. While it is a planned feature, at the moment there is no specific timeline for when it will be implemented. However, there are a number of resources available online that can help you build your own REST API to serve YOLOv8 model. One popular tool for building REST APIs is Flask, which can be used to create a web service that serves the YOLOv8 model.

[epochDVKHN]

Hi, So I have a small issue. I set the stream=True flag since I have a large video source but whenever I set this parameter as True, I do not see any video output being saved. However, setting the parameter as False saves the output but leads to a full RAM.

Here is the predict command used (in Google Colab): model.predict(source=VIDEO_SRC_PATH, stream=False, save=True, save_txt=False, device=0, conf=0.5, line_width=None, visualize=False, augment=True, agnostic_nms=True, project=PRED_VIDEO_DIR, name=f'Turtle_Video{VIDEO_CTR}')

Can you please provide any suggestions or modifications to be made? Thank you.

[apiszcz]

Are you setting this flag on the command line?

yolo cfg=test.yml stream=True

[epochDVKHN]

Are you setting this flag on the command line?

yolo cfg=test.yml stream=True

No, I'm setting this flag on a Jupyter Notebook cell, i.e., a Python statement. I even ran this on a local Jupyter Lab server (and similar to Google Colab), I see no output and the video is not being saved as well.

I'm running the latest version of Ultralytics on Python 3.10 kernel. (Should I open a new issue about this? Sorry I'm a newbie)

[apiszcz]

That is most likely that is the issue, i was trying to set stream in the configuration file and had the same issue. it needs to be set on the command line

On Sun, Jun 25, 2023 at 2:44 AM Dominic Alex Joseph < @.***> wrote:

Are you setting this flag on the command line?

yolo cfg=test.yml stream=True

No, I'm setting this flag on a Jupyter Notebook cell, i.e., a Python statement. I even ran this on a local Jupyter Lab server (and similar to Google Colab), I see no output and the video is not being saved as well.

— Reply to this email directly, view it on GitHub https://github.com/ultralytics/ultralytics/issues/1713#issuecomment-1605898450, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAK5KTNWYOSOCEWX4BFPJJDXM7M47ANCNFSM6AAAAAAWM4NETU . You are receiving this because you were mentioned.Message ID: @.***>

[glenn-jocher]

@ElNoSabe322 hi Dominic Alex Joseph,

Thank you for reaching out with your issue. From what you've mentioned, it seems like you're setting the stream flag in a Jupyter Notebook cell using a Python statement. However, in order to properly set the stream flag, it needs to be set on the command line, not in the configuration file or as a Python statement.

When running the YOLOv8 command, you can set the stream flag to True in the command line by using the following format:

yolo cfg=test.yml stream=True

This will enable the streaming mode and should allow the video to be saved without consuming excessive RAM.

I hope this explanation helps. If you have any further questions or need additional assistance, please don't hesitate to ask.

Kind regards, Glenn Jocher

[epochDVKHN]

That is most likely that is the issue, i was trying to set stream in the configuration file and had the same issue. it needs to be set on the command line …

@ElNoSabe322 hi Dominic Alex Joseph,

Thank you for reaching out with your issue. From what you've mentioned, it seems like you're setting the stream flag in a Jupyter Notebook cell using a Python statement. However, in order to properly set the stream flag, it needs to be set on the command line, not in the configuration file or as a Python statement.

When running the YOLOv8 command, you can set the stream flag to True in the command line by using the following format:

yolo cfg=test.yml stream=True

This will enable the streaming mode and should allow the video to be saved without consuming excessive RAM.

I hope this explanation helps. If you have any further questions or need additional assistance, please don't hesitate to ask.

Kind regards, Glenn Jocher

@apiszcz and @glenn-jocher, Oh I see... I was not aware of that 😮. I thought both CLI commands and Python statement parameters could be used interchangeably, but I guess the stream parameter is an exception. Thank you all so much. I appreciate your valuable explanations. ♥

[epochDVKHN]

@apiszcz and @glenn-jocher Sorry, but I now have another problem 😅. So, I took the suggestion provided and tried to run the stream=True parameter from the CLI but I now receive an error saying that 'stream' is not a valid YOLO argument.

Here's the code that I tried to run via Google Colab

# Predicting on videos
PredTimeStart = time.time()
VIDEO_CTR = 1
PRED_VIDEO_DIR = f'{HOME}/runs/detect/Detect_TestVideo_Predict'
for VIDEO_NAME in VIDEO_FILE_LIST[0:1]:
    VIDEO_SRC_PATH = os.path.join(VIDEO_DIR, VIDEO_NAME)

    # Running video prediction on CLI since the ```stream = True``` parameter works only 
    # with a CLI command and not as a Python statement
    !yolo cfg='/content/drive/MyDrive/Colab Notebooks/Detect/detect_predict_config.yaml' stream=True
    # model.predict(source=VIDEO_SRC_PATH, stream=True, save=True, save_txt=False, device=0, 
    #               conf=0.5, line_width=None, visualize=False, augment=True, agnostic_nms=True, 
    #               project=PRED_VIDEO_DIR, name=f'Turtle_Video{VIDEO_CTR}')

    VIDEO_CTR += 1
    # display.clear_output()
PredTime_Elapsed_1 = time.time()-PredTimeStart
print(f"\nPrediction on {len(VIDEO_FILE_LIST)} test videos completed in 
{PredTime_Elapsed_1/60:.0f} minutes and {PredTime_Elapsed_1%60:.0f} seconds.")

Here's the custom config file I made (yeah the extension should be .yaml) detect_predict_config.txt
And here is a screenshot of the error I received from Google Colab.

Sorry but yeah, I need some additional assistance. Without the stream parameter everything is working as intended. But with the stream parameter something goes wrong. Is there a mistake in the way, I'm calling the parameter perhaps?

[Laughing-q]

@ElNoSabe322 please use the latest version of ultralytics package.

[epochDVKHN]

@ElNoSabe322 please use the latest version of ultralytics package.

@Laughing-q I am using the latest version of the Ultralytics package (8.0.122).

Upon reading this whole thread once again, I realize that a solution was already provided which I did not check out. yolo predict cfg= and not yolo cfg.

Update: I tried !yolo predict cfg='/content/drive/MyDrive/Colab Notebooks/Detect/detect_predict_config.yaml' stream=True but I still get the same error from Google Colab. (Furthermore, I have specified mode: predict in the custom configuration file.)

[glenn-jocher]

@ElNoSabe322 I apologize for the confusion. It seems that the issue you are facing is not related to the stream parameter. In your updated code snippet, you are using yolo predict command instead of yolo cfg command, and have specified mode: predict in your custom configuration file. However, you are still encountering the same error.

To troubleshoot this issue, I would recommend checking the following:

1. Make sure that the custom configuration file detect_predict_config.yaml is correctly formatted and does not contain any syntax errors.

2. Verify that the file path for the configuration file is correct. Double-check if the file is located at the specified path /content/drive/MyDrive/Colab Notebooks/Detect/detect_predict_config.yaml.

3. Ensure that the Ultralytics package is installed correctly, and that you are using the latest version (8.0.122).

If the issue persists, I would recommend reaching out to the Ultralytics YOLOv8 repository for further assistance. They will be able to provide more specific guidance on how to resolve the error you are encountering.

I hope this helps. If you have any further questions, please let me know.