orbbec / pyorbbecsdk

OrbbecSDK python binding
https://orbbec.github.io/pyorbbecsdk/
Apache License 2.0
62 stars 20 forks source link

Playback from bag files - direct access to frames by index or one after the other #62

Open CBadam opened 1 week ago

CBadam commented 1 week ago

Hello, I am recording the color and depth streams for offline processing that takes a long time. To reproduce the recorded sequence, I am following the playback sample. https://github.com/orbbec/pyorbbecsdk/blob/main/examples/playback.py. However, my objective is to process every frame of the sequence, and using the example code and adding a processing function after "frames = pipeline.wait_for_frames(100)", I am experiencing a drop of frames (especially when the processing takes a while). Thus, I was wondering if there is a way to access the frames by index, one after one and not as if it is recording in real time. something like this: playback.set_real_time( false ); from realsense Thanks in advance!

hamamabrouk commented 1 week ago

Description: I am encountering the same issue with the playback mode of the pyorbbecsdk.Pipeline when processing .bag files. During playback, the pipeline continuously streams frames without the ability to pause or control the playback. This results in frames being lost or skipped when trying to save RGB images, depth images, and point clouds in real time.

Expected Behavior: The pipeline should provide a way to pause or control the playback (e.g., manual stepping through frames). This would allow enough time to process and save each frame (such as saving RGB images, depth images, and generating point clouds) without missing or skipping frames due to continuous playback.

Current Behavior: The pipeline continues playing and doesn't pause when the frames are being saved, causing some frames to be skipped. There is no explicit method like pause() or set_playback_speed(0) to manually control the playback or pause the pipeline during playback. Any attempts to stop or slow down the pipeline interrupts the playback entirely, making it difficult to process every frame properly. Steps to Reproduce: Initialize the pyorbbecsdk.Pipeline in playback mode with a .bag file. Attempt to read frames and save each frame as an RGB image, depth image, and point cloud. Observe that frames are skipped while saving due to the continuous playback.

Proposed Solution: Provide a pause() method or a set_playback_speed(0) functionality to allow manual control over the frame playback. Ensure that the pipeline can be paused without fully stopping, so that frames can be processed and saved without missing any frames.

Environment: SDK Version: v1.10.8 Platform: Ubuntu 22.04 LTS Hardware: Orbbec Femto Mega Camera

zhonghong322 commented 1 week ago

@hamamabrouk @CBadam Thank you for your suggestions. We will consider them in future versions of the Orbbec SDK to improve our recording and playback experience.