tokenize.c uses 98k of bss

[GoogleCodeExporter]

tokenize.c uses a lot of bss memory, which impacts memory footprint, even if 
you only run the decoder.

c:\on2\libvpx>size c:\on2\msys32\vp8\encoder\tokenize.c.o
   text    data     bss     dec     hex filename
   2272       0   98336  100608   18900 

These 2 arrays are large:

TOKENEXTRA vp8_dct_value_tokens[DCT_MAX_VALUE*2];
int vp8_dct_value_cost[DCT_MAX_VALUE*2];

vp8\common\entropy.h:#define DCT_MAX_VALUE           2048
typedef struct
{
    int Token;
    int Extra;
    const vp8_prob *context_tree;
    int skip_eob_node;
    int section;
} TOKENEXTRA;

Could you malloc these instead?
and/or reduce the size.

Original issue reported on code.google.com by fbarch...@chromium.org on 15 Sep 2010 at 11:38

[GoogleCodeExporter]

Moving these tables from bss to the heap doesn't change the overall memory 
footprint or the object size.. Unless your application is linking to the 
encoder, these symbols should be removed at link time for decode only 
applications:

jkoleszar@CPK-JOHNK-VM1 /c/on2/build-mingw
$ size ivf{enc,dec}.exe
   text    data     bss     dec     hex filename
 394656    5260  103032  502948   7aca4 ivfenc.exe
 188520    2768    3664  194952   2f988 ivfdec.exe

I think we could reduce the size of the arrays though.

Original comment by jkoles...@google.com on 16 Sep 2010 at 1:38

• Changed state: Accepted

[GoogleCodeExporter]

Fixed in https://review.webmproject.org/546

Original comment by jkoles...@google.com on 16 Sep 2010 at 4:44

• Changed state: Fixed

[GoogleCodeExporter]

This is going to help but Chrome will still have a problem of creating the bss 
memory for every tab that gets opened whether or not libvpx gets used ever. 
Chrome also needs the encoder now for Chromoting.

Original comment by fgalli...@google.com on 16 Sep 2010 at 5:48

[GoogleCodeExporter]

The difference between encode and decode memory size (ivfenc vs ivfdec) is on 
the order of 200-300k. If this 16k table is on the radar, it'd probably be 
better for us to work with chrome to find a solution where the encoder and 
decoder can be loaded separately on demand. This table may be reworked in Q4 
with the encoder overhaul too.

Original comment by jkoles...@google.com on 16 Sep 2010 at 6:41

[GoogleCodeExporter]

I think getting Chrome to change how they load DLLs is going to be a lot bigger 
task than changing the memory to be created dynamically. They are currently 
loading all dlls when a tab is created. So that is 32K of memory per tab that 
may never get used.

Original comment by fgalli...@google.com on 16 Sep 2010 at 7:16

[GoogleCodeExporter]

Does chrome load this lib before or after the tab is forked off? My point is 
that there may be things we can do to save a lot more than 16k per tab (what's 
the 32k you mention?)

There are a few reasons that we avoid mallocing global tables. iirc, these 
symbols were allocated on the heap at one time, and I moved them to bss. I'd 
rather shrink this table and move it to .rodata than move it back to the heap.

Original comment by jkoles...@google.com on 16 Sep 2010 at 7:52

[GoogleCodeExporter]

Original comment by albe...@google.com on 8 Mar 2012 at 12:10

• Added labels: Type-Bug
• Removed labels: Type-Defect