video file corruption protection

[RieGo]

hey otto and vasile. i don't know if anyone already thought about this, but is it possible to detect a recording start / record button press? if so, one could run a timer for i.e. 74 minutes and then force a recording stop (and possibly start a new recording) so we could avoid having people loose a 74 minute recording because of corrupted file header. just a quick idea. if it isn't possible, our only chance is to look for a way to fix the files.

[ottokiksmaler]

I guess it's doable, but it's not 74 minutes, or depends on the mode and fps. I should probably find a better way of detecting.

[RieGo]

or do you have any other idea how to fix it? the swap approach didn't work afaik. i'm in a testing mood right now :) but there's no hurry.

[Tommixoft]

There is already made this mod long time ago in NX-KS

[RieGo]

i thought NX-KS just has some kind of batch video recording? i'm thinking about when you start a recording normally, which 90% of people do, even with NX-KS batch recording available. so it's just meant as a failsave so people don't loose important recordings (again) it's definitely no high priority, but now as no time limit is more or less default, there are more and more people who don't know or forget about the limit...

[Tommixoft]

Yes you set 70mins, it stops recording after 70mins and continues, just what you want :) I dont use such things, i never film more than 4mins so no need, but from what i read it have things you want.

[vasile-gh]

I will eventually implement something along these lines - should not be difficult

[vasile-gh]

adopted as a request - closing the issue

[ottokiksmaler]

Can someone provide times at which the corruption occurs for affected modes? IIRC it's not the same for all modes.

[RieGo]

i just started testing. i'm going to test 1080p30/60/120 at first. what else do we need? probably pal 25/50/100. do you think we need to check all those different resolutions? afaik it's the same time frame on FHD and UHD. but maybe i'm wrong.

[ottokiksmaler]

IIRC it was around 74 minutes on uhd, not sure about 4k.

[vasile-gh]

If we make a list and I ever get the time to create that inter-process pipe so I am notified when recording starts, I could dynamically patch the 30-min limit and change it to the relevant "safe" limit.

Trouble is with those pesky users who insist I add features before I am ready :-)

So we could make Samsung restrictions work for us for once :-)

[Tommixoft]

@ottokiksmaler , @vasile-gh -what causes file corruption? i heard that memory leak? Maybe it's possible to call some GarbageCollector so it clears memory. Dont know is there such thing in linux. This could help. Or atleast extend recording time.

[ottokiksmaler]

I already have a method of restating video on certain second but I need a time limit to implement.

[Tommixoft]

You being doctors, who not threat but just prescribe medications :D Problem memory limit, so why not just clear memory? linux should have such "app" to unload not used modules, memory. Looks like windows have some advantages in some cases :D

[ottokiksmaler]

Tool that fixes memory leaks is in the same package with tools that prevent deadlocks, buffer overflows, pointer related bugs and PEBKAC errors.

[Tommixoft]

Nieko visiškai nesupratau ką parašei :D - Yes exacly - i understood the same amount of info as you in my first sentence :) ANd it even dont answer anything..so this is good or bad? :D

[RieGo]

it doesn't exist :) i guess i will make a google spreadsheet or something with my test results. just give me some hours. jfyi: https://docs.google.com/spreadsheets/d/1OrhiUiF3s0dTzK6MMa04DtaglZBAL8YHVAbJ_7BhKTc/edit?usp=sharing i will add results while i'm doing the tests. gonna take a while. definitely not done by today

[RieGo]

so i'm done with my first tests (see spreadsheet) and i think i found a formula to calculate time limit based on fps:

MaxSeconds=Min(4608;216000/FPS)

it seems like there is a hard time limit (~76:48) + a frame count limit (~216000) which breaks recordings. tomorrow i will check if it makes a difference to turn on everything possible while recording. if i get the same results as with my blank video + everything on manual test i guess this formula is confirmed :) maybe results on nx500 will vary.

[vasile-gh]

I also think I read somewhere that for no-sound there's no limit but I may be mistaken. I remember that at the time I said to myself that if I were to search for the memory leak I would do it around the sound encoder.

[RieGo]

will definitely check that tomorrow

[ottokiksmaler]

From log files, the leak was most probably caused by audio, at least it was triggered by trying to allocate an audio frame in memory (that was in three log files I examined - doesn't have to be true, but it's highly likely). I have made a proof of concept code that parses log file in real time and just does a stop and start of recording (via st key click rec) triggered by current recording duration and it works.

I have to test it some more (had a segfault but most probably unrelated). If Vasile gets better communication with di-camera-app it will be even better.

[RieGo]

just finished my SlowMotion test, which has no audio. FHD 30 1/4 =120 fps. same result as "normal" FHD 120 fps. recording fails after 30:02 (real time).

[ottokiksmaler]

You should have several files with timestamp of your video file on SD card (log crash, etc) could you upload them somewhere (pastebin or wherever)?

[vasile-gh]

@ottokiksmaler please make sure you test the mod with my nx-patch - I am doing my best to kill all logging turning off everything I can w/st. Doing it to reduce as much as possible drop frames. and to maximize possible bitrates.

[ottokiksmaler]

Ouch, if there is no /dev/log_main It won't work

[RieGo]

here you go: https://drive.google.com/file/d/0B5oN6i68oB0RZzQ3Um5OT0lpMVk/view?usp=sharing

[just a thought: maybe audio is responsible for 76minute limit and video or something else for frame count limit]

[ottokiksmaler]

And this is with Vasile patch that disabled logging?

[RieGo]

yes it is! didn't think about that... do you need a log without the patch running? is it the nx-patch or the nx-on-wake mod which disables logging?

[ottokiksmaler]

No, this is great, even with logging disabled there still is a log :) that I need. But, there is no mention of error present is logs from crashed videos with audio so it's likely that audio was just a false flag or trigger that did not actually cause the crash.

[RieGo]

alright :). then maybe the audio encoder just crashed because it was running out of memory.

[vasile-gh]

@ottokiksmaler looks like I need to start digging to find more logs to disable :-)

[vasile-gh]

@ottokiksmaler I disable logs in nx-patch not in nx-on-wake. I kept nx-on-wake very focused on doing one thing only :-)

[ottokiksmaler]

I don't think logs have anything to do with dropped frames, writing to log is done in Linux and encoding is done by other cores. You would have to disable it within t-kernel to have some effect on frames.

FWIW, while reading and parsing the log my program uses under 0.1% of cpu as top reports 0.0% unless I'm also doing something else (calling st and the like).

[RieGo]

after 3 days of testing limits i can now say i fairly certain know how the "limit" works: there seem to be two kinds of bugs/memory leaks(?)

• (probably audio) error after ~1:16:48
• (video or something similar) error after ~216.000 frames

so whatever limit is met first stops the recording and corrupts the file. i summarised my findings here: https://docs.google.com/spreadsheets/d/1OrhiUiF3s0dTzK6MMa04DtaglZBAL8YHVAbJ_7BhKTc/edit?usp=sharing

the last recording was 10 hours... so i allow my cam some rest now :)

[ottokiksmaler]

Thank you so much, but 126k or 216k frames? I think 216k?

[RieGo]

you are right. i mistyped. :)

[ottokiksmaler]

OK, here is a simple tool that watches the system logs, parses the output and restarts the video recording after set number of seconds and/or frames (whatever comes first). It also monitors what files are displayed and/or selected in image browser and generates files with their filenames when sent USR1 signal (can be used in future for upload to other services, etc, use it like this killall -SIGUSR1 @log_watcher).

File is here, instructions to test tools here.

I have tested on NX500 with shorter time/frame spans and it worked as it should in 24/30/60 fps. My camera generally dislikes 120fps mode (takes very long to actually stop recording, but it did stop recording cleanly the first clip and started the second).

Regarding resource usage - RAM is minor, CPU is 0.0% in normal operation, 0.3% while calling st key click rec or creating files, according to top output.

Testers wanted :)

---

Tool to restart video after set number of seconds or frames.

Usage: log_watcher help - show this help log_watcher [max_seconds] [max_frames] [debug|debug_all] - default 4500 s and 215000 frames

This tool also monitors selected files in gallery and creates a file with selected filenames when sent USR1 signal: /tmp/log_watcher_current_open_file - with the filename of currently displayed file /tmp/log_watcher_selected_files - comma separated filenames of selected files.

[Tommixoft]

Just pls dont implement things like that in general mod. Writing logs to flash memory is bad idea. and better corrupted video than constant writes. Writes SLOWS things as hell. Always was and always will.

[ottokiksmaler]

Ok, first, I'm not writing, I'm reading. Second - the log file is not a regular file, it lives in /dev and is filed by system. Ave files are created in tmpfs which lives in RAM.

[outerbeat]

I need to mention that now we have the way to fix corrupted HEVC file with unreleased yet but functional utility.

Here is h264 version utility http://slydiman.narod.ru/eng/mmedia/recover_mp4.htm You can connect with author via email on his page and ask to use h265 version. It works fine, little audio lag that you can manage in any software, but I was able fully recover h265 file UHD@30 which was crashed due record limit fail. All you need is that crashed file and good one from same session.

[RieGo]

thanks a lot for this. i will test this soon. hopefully it get's integrated in a mod :) regarding concerns: it surely will not slow down anything imho. as we know it runs on a different core as video/photo stuff and i'm also sure otto made it very light. also sending in 100gb files to a guy to fix it (in future maybe not even free) seems not to be a good solution in this case. even though we are very thankful that he is able to do this. this is the only clean fix/workaround to prevent new people from this trap.

uploading files to services will be awesome. can't wait to try it out. i think samsung planned to integrate something like this, right? there are many strings like google drive, flickr, youtube, etc in the fw.

[outerbeat]

also sending in 100gb files to a guy to fix it (in future maybe not even free) seems not to be a good solution in this case

Sure, it's not best way to deal with this. But here is his point - he did the work on H265 recovery even before we find him, but not fully implement it because he doesn't have any h265 crashed file at the time, we provide him such files and he finalize the app, But his intention was to setup online platform for recovery, not distribute the program in any way, so... I told him main reasons of that crashes in our cases, so he knows that files going to be huge, and he take time to think about. We'll see :)

[Tommixoft]

@ottokiksmaler how your protection works? Looking at time of recording is bad approach. What if camera have some big mods who uses RAM, your times based on ram from camera with your mods. What if other mods uses 200MB ram. does time still remains the same while it not corrupts?

[ottokiksmaler]

I honestly have no idea what you asked.

[Tommixoft]

AFAIK file corruption happens because of memory leak - full ram. So problem is not time itself but free ram. And i ask-what will happen if i alocate-use 300MB ram (lets say mod uses that much), does your corruption protection will work? cause i doubt that i then could film 70minutes.Ram should be wasted earlier than that.

[ottokiksmaler]

It's not that simple, you can allocate enough memory for what you need (here a 216000 frames of some data) and you test it and it works for anything up to 30 minutes (30_60_120=216k). But when we push it over that limit it breaks apart. There is no guarantee that actual RAM is exhausted, maybe just some buffer fills.

Also, I'm reading a log file that's not an actual file but looks as one, I'm not measuring RAM since in several tests done by outerbeat and riego it turned out that even without audio there is still a corruption - so it's not only audio, it might be unrelated but just exposed by audio.

[Tommixoft]

OK understood now, thanks :)

[RieGo]

i almost forgot about this :smile: quick test (10 seconds) on NX1: stopping the recording works! and it seems to wait for a second. then the camera gui just crashes/restarts. i tried entering the command manually while recording

st key click rec && sleep 2 && st key click rec

and then stopping and starting works. seems like NX1 doesn't like something in this tool... anyways the recording is NOT broken. so that's a good start :+1: