search

MikaTake / lavfilters

lavfilters
GNU General Public License v2.0
0 stars 0 forks source link

Configurable memory limit for the splitter queues. #274

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
When playing back 3 or 4 Apple ProRes 1920x1080 mov files on WIndows 7 32-bit 
the system runs out of memory and LAV splitter crashes in BaseDemuxer.h 
function: SetDataSize. CoTaskMemRealloc fails and after that an invalid pointer 
(0) is used to init the memory block without checking the pointer. This causes 
access violation.
It looks like AVsplitter keeps buffering packets until the memory runs out.
The crash does not happen on 64-bit Windows due to a larger virtual memory 
availability.
Is there a way to fix this maybe by limiting the sample buffer queue size or 
checking the pointer after CoTaskMemRealloc?

Original issue reported on code.google.com by paveltbo...@gmail.com on 1 Oct 2012 at 9:18

GoogleCodeExporter commented 9 years ago 
LAV Splitter already has a memory queue limit of 192MB.
To make it run out of memory, you need to run several videos simultaneously in 
the same process. If you just run multiple processes, it would go into virtual 
memory and hit the swap.

So either you have no swap, or you're doing something terribly wrong.
The 192MB limit should be fine for like 99.9% of all use-cases.

Original comment by h.lepp...@gmail.com on 1 Oct 2012 at 10:18

GoogleCodeExporter commented 9 years ago 
You are right - it happens in the same process.
The test was performed with Advanced Graph Manger dropping the same file in
few times and playing simultaneously. We thought we may be doing something
wrong, so we tested with different software.
There is a swap file but it also runs out because of the 32-bit limitation
(Used Resource Monitor system tool).

Is it at least possible to put a check for the mem pointers in
BaseDemuxer.h and not memset or memcpy when they are null after
CoTaskMemRealloc?
We don't care about dropping frames.
We have to playback multiple clips simultaneously in the same app  (
www.motionrocket.com).

Original comment by paveltbo...@gmail.com on 1 Oct 2012 at 11:09

GoogleCodeExporter commented 9 years ago 
Dear LAVfilters developer!

We understand that it is time consuming and difficult to support the
project. We are willing make a small donation in exchange for your time.
We would like to make a request for an option to control the source samples
queue depth in LAV splitter configuration.
Please let us know if that is something you would be interested in.

Best regards:
Pavel Bonev
MotionRocket LLC
Owner/CTO

Original comment by paveltbo...@gmail.com on 2 Oct 2012 at 2:41

GoogleCodeExporter commented 9 years ago 
I added some checks for the allocations for the next version, but i cannot 
guarantee that it will not crash anymore - out of memory conditions in a 
DirectShow system are rather unpredictable.

I can add an option to control the maximum queue size, if you think that can 
help you.

Original comment by h.lepp...@gmail.com on 2 Oct 2012 at 6:14

GoogleCodeExporter commented 9 years ago 
We greatly appreciate your response.
I think adding an option to control the queue size will greatly increase
our ability to control resources and state the maximum capabilities of the
system.
I can offer you $500 to implement this as an option and of course this will
be a contribution to the project, not a custom request.

Best regards:
Pavel B.

Original comment by paveltbo...@gmail.com on 2 Oct 2012 at 6:39

GoogleCodeExporter commented 9 years ago 
I'll see about how to add it, and when i can get to it. Thanks for the generous 
offer.

Original comment by h.lepp...@gmail.com on 2 Oct 2012 at 6:56

GoogleCodeExporter commented 9 years ago 
We sent you a donation of $500 as we promised. I did not see the link to
donate yesterday for some reason.
Thanks for you efforts to help!

One favor I need to ask - can you build LAV filters for x86 and x64 with
the latest change(s)?
For some reason when I build them the LAVdecoder would not register and the
LAVsplitter us unstable. probably something is different with my
environment.

Original comment by paveltbo...@gmail.com on 2 Oct 2012 at 11:25

GoogleCodeExporter commented 9 years ago 
Hi,

here are the two builds:
x86: http://files.1f0.de/lavf/LAVFilters-0.51.3-95-ga6b7a05.zip
x64: http://files.1f0.de/lavf/LAVFilters-0.51.3-95-ga6b7a05-x64.zip

Note that these are still development builds, but hopefully they should work 
just fine otherwise.

There is now a new option included which lets you configure the memory limit of 
the queues, so you can reduce it if that better suits your needs. Note that its 
just a limit, there are other factors which control how much memory it uses, so 
if you increase it, it may not use any more memory - but i think you were 
mostly interested in reducing it, anyway. :)

Original comment by h.lepp...@gmail.com on 4 Oct 2012 at 8:31

GoogleCodeExporter commented 9 years ago 
Thanks a lot!
We will be testing it today and tomorrow and will let you know!

Best regards:
Pavel B.

Original comment by paveltbo...@gmail.com on 4 Oct 2012 at 9:32