facebookresearch / vrs

VRS is a file format optimized to record & playback streams of sensor data, such as images, audio samples, and any other discrete sensors (IMU, temperature, etc), stored in per-device streams of timestamped records.
https://facebookresearch.github.io/vrs/
Apache License 2.0
323 stars 55 forks source link

How to play back VRS files at multiple speeds? #151

Closed buyibo64 closed 3 months ago

buyibo64 commented 3 months ago

I want to know how to customize the playback speed when playing VRS files, but I couldn't find the relevant interface definition in the API documentation. Thank you very much!

georges-berenger commented 3 months ago

Sorry for not seeing your question sooner. VRS playback doesn't follow any timeline by default: your read records, which happens to have timestamps. Reading a record will take "some time", depending on your system's performance, where the data is stored, cached, etc. If after reading a record, your code will do some some data processing (like decoding images, propagating the record's data where you need it to go), and then your code will probably just read the next bits (presumably the next record). Depending on how much processing your code has been doing, like decoding jpg data in the same thread maybe, you might be faster or slower than the timestamps of the VRS data you're reading. But VRS won't mind: it won't slow you down if you go too fast, or skip data if processing is too slow, vrs will just try to execute every operation as fast as it can. Now, vrsplayer, the app, tries to track the record's timestamps, offloading image decoding on different threads, reading files in a background thread, while obviously displaying image in the UI thread, possibly skipping records if all that multithreading isn't enough to keep up. It also offers options to speed or slow down playback. But vrsplayer isn't vrs: it's an app which plays back VRS data at different speed, 1x by default. In other words, your code is entirely responsible for controlling playback speed, and by default, your code will probably just try to go as fast as possible, just reading one record after the other, until the end of the file.

buyibo64 commented 3 months ago

Thank you very much for your detailed and professional explanation! I am currently studying the vrsplayer source code and hope to gain more insights and inspiration from it. If I have more questions, I will reach out to you again. Thanks again!