Packet timestamp mismatch - mediapipe does not recover

[maikthomas]

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

Yes

OS Platform and Distribution

macOS 14.6.1

MediaPipe Tasks SDK version

I tested with

• @mediapipe/tasks-vision 0.10.18
• @mediapipe/tasks-vision 0.10.15

Task name (e.g. Image classification, Gesture recognition etc.)

ImageSegmenter

Programming Language and version (e.g. C++, Python, Java)

Javascript - Web

Describe the actual behavior

mediapipe throws the same error with the same timestamp when given new frames. It also offers no method to reset the timestamp.

This issue is concerning error below. I may open an issue as to why the error is occurring but for now I want to address the fact that mediapipe does not recover from this error. I forked and adapted a codepen to show this behaviour: https://codepen.io/maikthomas/pen/GRVbWNb In this pen, when using the webcam, on the 11th frame I manually use a timestamp greater than the previous timestamp. I then return to using the correct timestamps. However mediapipe will continue to report the same error with the timestamps from before.

Another possible case is when it is known that the timestamp may be lower, due to different input sources (some timestamps start at 0). To avoid this I am currently using Date.now(). But it would be good to be able to reset the current minimum expected timestamp back to 0 if needed.

Describe the expected behaviour

mediapipe uses the newly provided timestamp and recovers. And also exposes a method in order to reset timestamp manually

Standalone code/steps you may have used to try to get what you need

As a workaround for now I am calling segmentForVideo in a tr/catch and recreating the ImageSegmenter when it fails

Other info / Complete Logs

vision_wasm_internal.js:10 E1120 09:57:18.478000 1895072 gl_graph_runner_internal_image.cc:68] Adding Image to stream image_in was not ok: INVALID_ARGUMENT: Graph has errors: 
Packet timestamp mismatch on a calculator receiving from stream "norm_rect". Current minimum expected timestamp is 1732096638495001 but received 1732096638472000. Are you using a custom InputStreamHandler? Note that some InputStreamHandlers allow timestamps that are not strictly monotonically increasing. See for example the ImmediateInputStreamHandler class comment. [type.googleapis.com/mediapipe.StatusList='\n\xf5\x02\x08\x03\x12\xf0\x02Packet timestamp mismatch on a calculator receiving from stream \"norm_rect\". Current minimum expected timestamp is 1732096638495001 but received 1732096638472000. Are you using a custom InputStreamHandler? Note that some InputStreamHandlers allow timestamps that are not strictly monotonically increasing. See for example the ImmediateInputStreamHandler class comment.']
=== Source Location Trace: ===
third_party/mediapipe/framework/input_stream_manager.cc:159

[kuaashish]

Hi @maikthomas,

We have newer version available 0.10.18, Can you try and let us know the status now?

Thank you!!

[maikthomas]

@kuaashish apologies this was a mistake filling out the form, I have already tried with 0.10.18. I was using 0.10.15 and upgraded to 0.10.18 and saw the same issue. I'll update this info the issue description too.