search

FFMS / ffms2

An FFmpeg based source library and Avisynth/VapourSynth plugin for easy frame accurate access
Other
582 stars 105 forks source link

Very obscure issue! #201

Closed ScHlAuChii closed 9 years ago

ScHlAuChii commented 9 years ago

Some info: 164 AVI files in ZMBV codec (DosBox default) with WAV as audio.

Loading those files with AviSynth looks like this: FFCopyrightInfringment("I:\DOSBox\capture\mm_000.avi").AssumeFPS(70).Trim(0, 109) ++ FFC etc
Everything seems fine, but once I resave everything in a lossless codec like Lagarith, i end up with multiple errors like this: http://realmods.online.fr/MM.gif

Now this is where things get weird. Of course I checked the source avis if they are broken in any way, and they are not - so I tried opening the exact same files without audio via ffmpegsource2("I:\DOSBox\capture\mm_000.avi").AssumeFPS(70).Trim(0, 109) ++ ffm etc Resaving it this time results in a perfectly fine video!

I still have all the files and the scripts in case it helps with debugging this. The best way to contact me is via the IRC chat here - nickname ScHlAuChi: http://longplays.org/viewpage.php?page_id=9

Thanks!

myrsloik commented 9 years ago

Did you try with other lossles codecs than Lagarith? It's known to be conceptually broken and corrupt video now and then.

ScHlAuChii commented 9 years ago

Yep I tried - same result. It even happens when resaving as Uncompressed. So I dont think it has anything to do with the codecs used during the resaving. When I open the files with ffmpegsource2 and then resave with Lagarith & Co the issue does not happen. Opening the files with ffmpegsource2 and .atrack=1 results in the same broken video. I packed up all the files and scripts into a 132MB 7z file. It can be downloaded from our FTP (contact me in IRC for details)

ScHlAuChii commented 9 years ago

In case you dont have the DosBox ZMBV codec, I uploaded it here: http://realmods.online.fr/ZMBV64.7z

ScHlAuChii commented 9 years ago

So, were you able to reproduce the issue?

myrsloik commented 9 years ago

Not the issue you described. But it did have an access violation the first time I tried to process it. Maybe that's connected...

myrsloik commented 9 years ago

This is what I suspect happens: Since you're processing with 32bit vdub it hits the 2GB address space limit (it's always very close) and a memory allocation fails inside avisynth. However avisynth is terrible code in many ways and catches this exception and pretends it's raining. From this point in time you get corrupted junk in your output.

Sprinkle a SetMemoryMax(256) at the top of your script and it's less likely to happens or use avs+ and 64bit stuff which should also work.

I think the reason you reach the limit is that that shitload of ffms2 instances allocate quite a bit of memory plus the 1GB or so used for cache by avisynth and the disk cache vdub has which probably turns unusually big when the harddrive is the lmiting factor.

SailorOnDaTea commented 9 years ago

I think this is related: When I was trying to play some 30MB real video via avs, mpchc would show an error message about access violation instead of video. After googling, I found that I needed to set how much memory to use in script. In my case any value below around 400MB would fail. I mean, it is probably true that it is a memory allocation/access issue. If it is 32bit, 2GB can be bottleneck for large videos.

--- Original Message ---

From: "Fredrik Mellbin" notifications@github.com Sent: May 1, 2015 8:11 PM To: "FFMS/ffms2" ffms2@noreply.github.com Subject: Re: [ffms2] Very obscure issue! (#201)

This is what I suspect happens: Since you're processing with 32bit vdub it hits the 2GB address space limit (it's always very close) and a memory allocation fails inside avisynth. However avisynth is terrible code in many ways and catches this exception and pretends it's raining. From this point in time you get corrupted junk in your output.

Sprinkle a SetMemoryMax(256) at the top of your script and it's less likely to happens or use avs+ and 64bit stuff which should also work.

I think the reason you reach the limit is that that shitload of ffms2 instances allocate quite a bit of memory plus the 1GB or so used for cache by avisynth and the disk cache vdub has which probably turns unusually big when the harddrive is the lmiting factor.

Reply to this email directly or view it on GitHub: https://github.com/FFMS/ffms2/issues/201#issuecomment-98179675

myrsloik commented 9 years ago

Sounds more and more like an avisynth issue. Can't fix that.

ScHlAuChii commented 9 years ago

So the reason why it only happens in combination with the audio is because it goes over the 32bit adress space limit, and without audio its still a little below it, which is why it didnt happen there?

And yeah if its AviSynth´s fault then nothing you can do of course - I didnt know what exactly caused it and just thought it would be ffms2 - thanks anyway for taking the time looking into it :)

Also thanks for making me aware of AviSynth+ will give it a try :)

myrsloik commented 9 years ago

Another way you could try to verify it is by using a 32 bit virtualdub but with largeaddressaware set which can give it a bit more margin. But going 64bit is better in general anyway...