Thread abstraction makes no attempt to set stacksize resulting in larger virtual memory footprint

[GoogleCodeExporter]

What is the expected behavior? What do you see instead?
When threads are created, a stacksize should be set because the default is 
often much too high
On linux 64 the thread default stack size is 8mb when using lots of CPU this 
adds up to many dozens of megs of virtual ram to be used.

What version are you using? On what operating system?
git version c77d4dcb355798870bde4e759a547203e899bcc9 on linux and mac

Can you reproduce using the vpxdec or vpxenc tools? What command line are
you using?
No it was reproduced inside a multi-threaded app using params to set more 
threads but it should show the same results with vpxenc and vpxdec

Please provide any additional information below.

I think the threading abstraction is too light in vp8.  I see that there is a 
slightly more substantial one in vp9 but it makes the same mistake to try to 
convert non-pthreads into pthreads leaving no room for abstraction of the attrs 
making it impossible to properly have abstraction of stack size and other 
params.

Ideally a generic abstraction should be used so each supported platform can 
implement threading functions and how to set various attributes like stack size.

I attached a patch to call thread_create instead of pthread_create so linux and 
mac could force a stacksize of 240k.  This is more of an example than a 
solution.  Really more care should probably be put into portability so the 
threaded mode of operation can function properly and save resources rather than 
consume more.

Original issue reported on code.google.com by anthony....@gmail.com on 24 Mar 2015 at 8:29

Attachments:

• vpx.thread.diff

[GoogleCodeExporter]

> I think the threading abstraction is too light in vp8.

True. It should be considered legacy at this point.

> I see that there is a
> slightly more substantial one in vp9 but it makes the same mistake to try to
> convert non-pthreads into pthreads leaving no room for abstraction of the
> attrs making it impossible to properly have abstraction of stack size and
> other params.

Note vp9_set_worker_interface() [1] allows you to override the thread worker 
implementation.

[1] 
https://gerrit.chromium.org/gerrit/gitweb?p=webm/libvpx.git;a=blob;f=vp9/common/
vp9_thread.h;h=12848fedeff0624b91b81a489f7bfcec31492829;hb=5c78c2f19eec4eb78d866
79b996231218f551022#l212

Original comment by jz...@google.com on 24 Mar 2015 at 9:43

[GoogleCodeExporter]

I understand that its more important to focus on vp9 but for a long time going 
forward the current master of vp8 will probably end up into distros and 
imprinted for years with the performance issues.

So is it worth trying to improve so we will not have to urge people using these 
platforms to build their own version to overcome these issues?

Also is it worth making the default vp9 try to use smaller stack size?  at 
least we can override it so I am least worried about that.

Original comment by anthony....@gmail.com on 24 Mar 2015 at 9:52

[GoogleCodeExporter]

I didn't mean to discount the other points, yes this should be looked at; I was 
just pointing out that there were other options in the meantime.
Ideally vp8/9 would share the same threading code making any updates simpler.

Original comment by jz...@google.com on 24 Mar 2015 at 9:56

[GoogleCodeExporter]

Original comment by ya...@google.com on 2 Apr 2015 at 9:41

• Changed state: Assigned

[GoogleCodeExporter]

Original comment by jimbankoski@google.com on 8 May 2015 at 1:31