Android StopRecording fails if scene with VideoKitRecorder in get's unloaded and reloaded.

[supermoos]

I have Scene with a VideoKitRecorder gameobject in it (see screenshot). The first time I run and load this scene on Android it works fine, and saves the recording correctly. I then unload the Scene, go to another scene and reload the scene with the VideoKitRecorder gameobject in it again, the second time this scene get's loaded and I call StopRecording, Android starts spitting out a bunch of errors (see screenshot).

Note, the VideoKitRecorder gameobject is disabled by default, and I call SetActive(true) on it with a small delay so I can control when the prepareRecorder logic starts running, as it stalls the scene for a sec or two.

[olokobayusuf]

Hey @supermoos can you try disabling the Prepare on Awake boolean then trying again? Let me know if the issue persists.

[supermoos]

Then it doesn't work at all.

[olokobayusuf]

What device and OS version are you recording on? Can you share the full logs from app start in a .txt attachment? Make sure to capture all logs (no filtering).

[olokobayusuf]

This looks to be device-specific at first glance.

[supermoos]

Disabling prepare on wake worked at least one time now. Here's the log with prepare on wake enabled. log.txt

[supermoos]

log_no_prepare_on_wake.log Here's one that failed even though prepare on awake was not checked.

[olokobayusuf]

What device and OS version is this? This issue is device-specific:

Warn CCodec previous call to queue exceeded timeout
Error MediaCodec Codec reported err 0x80000000, actionCode 0, while in state 6

One last thing to try is to record at 1280x720 and see if that helps.

[supermoos]

It's a Google Pixel 4, Android Version 11

[olokobayusuf]

Try out 1280x720 and let me know if that has the same behaviour.

[supermoos]

1280x720_prepare_on_wake.log It works when prepare on wake is not checked, however when it is checked it only works the first time the scene is loaded, if I unload it, and load it again the above log errors appear.

[olokobayusuf]

This is an device-specific bug. From the logs:

Warn CCodec previous call to queue exceeded timeout

Then everything from there fails and the errors cascade. We have to improve error handling so that the logs don't get spammed with IllegalStateException, but that isn't the actual issue here. I'm closing this issue.

[olokobayusuf]

Minor note on this: NatCorder doesn't actually have a timeout for how long it's willing to wait for the encoder. We ask to wait infinitely until the encoder is ready to give us a buffer to use.

[supermoos]

Okay, well is there some workaround maybe? It's also an issue on another device I tested on (pixel3a, android 10, api 29). Log attached. pixel3a.txt

[supermoos]

Just did another test on a Lenovo YT-J706f android 11 api 30 - it also throws a bunch of errors and outputs a completely black video. lenovo_yt_j706f_android_11_api_30.txt

[supermoos]

The second time I try a recording on the Lenovo it works.

[supermoos]

And here's another failed test on a Pixel 4A, Android 13, SDK 33. pixel4a_android13_sdk_33.txt

[supermoos]

It works fine on a Pixel 7. However that's a lot of devices that don't work / work very flaky - I'm not entiely sure I would write it off as a device issue without at least providing better error handling, perhaps including some fallback configurations that could be retried. My understanding was that MP4 recording and api level 24+ was suppose to be the only minimum requirements for it to work? But maybe I'm missing something? :-)

[olokobayusuf]

Okay, well is there some workaround maybe? It's also an issue on another device I tested on (pixel3a, android 10, api 29). Log attached.

Same error:

Warn CCodec previous call to queue exceeded timeout
Error MediaCodec Codec reported err 0x80000000, actionCode 0, while in state 6

Just did another test on a Lenovo YT-J706f android 11 api 30 - it also throws a bunch of errors and outputs a completely black video.

Info MPEG4Writer setStartTimestampUs: 105257
Info MPEG4Writer Earliest track starting time: 105257
Warn MPEG4Writer Condition timestampUs >= 0LL failed for Video track
Error MPEG4Writer Dumping Video track's last 1 frames' timestamps(pts, dts) and frame type : 
Error MPEG4Writer (0us, 0us Key frame) 

This seems to be complaining about negative timestamps, but that can't be the case since timestamps generated for recording are guaranteed to be monotonic. The only potential issue, which is also extremely unlikely, is if pixel buffers are being committed out of order. One way to test this is to set your app to render with OpenGL ES3 in Player Settings, then add the following code before you start recording:

NatML.Recorders.TextureInput.UseGLESTextureInput = false;

And here's another failed test on a Pixel 4A, Android 13, SDK 33.

Same as the other Pixel errors:

Warn CCodec [c2.qti.avc.encoder] previous call to queue exceeded timeout
Error MediaCodec Codec reported err 0x80000000/UNKNOWN_ERROR, actionCode 0, while in state 6/STARTED

---

I can't help but notice that for the 4-5 seconds where you record, there are no logs being generated:

2023/05/03 **10:01:38.048** 9589 14287 Info MPEG4Writer Earliest track starting time: 0
2023/05/03 **10:01:42.452** 9589 9617 Info Unity Stop recording

Are you filtering logs? If so, please refrain from doing that cos I need to see everything.

[supermoos]

I haven't configured any filtering. The logs are from the Unity Android Logcat window. The filtering is set to verbose?

I just ran another test on pixel 7 and this time it started throwing errors and produced a fully black video.

Logs attached. pixel7_error.txt

[olokobayusuf]

Same error as the Lenovo:

Info MPEG4Writer setStartTimestampUs: 5126
Info MPEG4Writer Earliest track starting time: 5126
Warn MPEG4Writer Condition timestampUs >= 0LL failed for Video track
Error MPEG4Writer Dumping Video track's last 1 frames' timestamps(pts, dts) and frame type : 
Error MPEG4Writer (0us, 0us Key frame) 

Did you try out what I mentioned above?

[supermoos]

Do you mean like the screenshot? I'm getting a compile error with the code snippet you provided: Assets/game_doodles/Scripts/GameManager.cs(133,13): error CS0234: The type or namespace name 'TextureInput' does not exist in the namespace 'NatML.Recorders' (are you missing an assembly reference?)

Let me know if it's easier to jump in discord on a call or something.

[olokobayusuf]

Oops, missed a namespace:

NatML.Recorders.Inputs.TextureInput.UseGLESTextureInput = false;

[supermoos]

Yes that seems to work on the pixel 7 at least the two times I've tested now. Here's the log. However I'm not. sur eof the implication of switching the graphics API for my app. Shouldn't it work without this non-default setup too?

pixel7_es3.txt

[supermoos]

It also works on the lenovo. lenovo_working_es3.txt

[supermoos]

The pixel 4 still doesn't work however. pixel4_es_not_working.txt

[supermoos]

The pixel 4a worked on the first run, second run failed. pixel4a_second_run_error_es3.txt

[olokobayusuf]

Yes that seems to work on the pixel 7 at least the two times I've tested now. Here's the log. However I'm not. sur eof the implication of switching the graphics API for my app. Shouldn't it work without this non-default setup too?

This means that the pixel readback buffers are likely out of order. Your app was rendering with Vulkan before, which means that it's a bug from Unity's implementation of AsyncGPUReadback. The readbacks don't seem to be coming back in the same order as they were requested.

The pixel 4 still doesn't work however.

Same error:

Warn CCodec previous call to queue exceeded timeout
Error MediaCodec Codec reported err 0x80000000, actionCode 0, while in state 6

The pixel 4a worked on the first run, second run failed.

Same error:

Warn CCodec [c2.qti.avc.encoder] previous call to queue exceeded timeout
Error MediaCodec Codec reported err 0x80000000/UNKNOWN_ERROR, actionCode 0, while in state 6/STARTED

[olokobayusuf]

TL;DR: For the Pixel 4s, it's device-specific. And for the Pixel 7 and Lenovo, it's a Unity bug.

[supermoos]

Okay, is it something you're planning to add work arounds for, since effectively you're solution (we have a subscription for) is not working on a whole bunch of Android devices? Happy to help out in any way I can with debugging.

[olokobayusuf]

Okay, is it something you're planning to add work arounds for.

We might be able to add a workaround for the timestamp issue on the Pixel 7. As for the Pixel 4, there isn't anything we can do; it's a device-specific issue.

since effectively you're solution (we have a subscription for) is not working on a whole bunch of Android devices

We don't have any other reports of this, so it seems "whole bunch of Android devices" only refers to the Pixel 4 series. Except you have more?

[supermoos]

The Pixel 3A also exhibits the same issue. Would be great with a workaround for the pixel 7, and perhaps better error handling for the devices you're unable to support.

[olokobayusuf]

Added workarounds for Pixel 7 in latest VideoKit alpha build.. I'm closing this issue.