search

Chen-tao / webm

Automatically exported from code.google.com/p/webm
0 stars 0 forks source link

Missing rounding in rdmult calculation #576

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 8 years ago 
In "vp8_initialize_rd_consts()" function of Rdopt.c, 

void vp8_initialize_rd_consts(VP8_COMP *cpi, int Qvalue)
{
    int q;
    int i;
    double capped_q = (Qvalue < 160) ? (double)Qvalue : 160.0;
    double rdconst = 2.80;

    vp8_clear_system_state();  //__asm emms;

    cpi->RDMULT = (int)(rdconst * (capped_q * capped_q));

........

}

This can give unexpected results that vary across platforms (see capped_q = 
55.0 for example). Originally reported by G.Balaji here:

https://groups.google.com/a/webmproject.org/forum/?fromgroups=#!topic/webm-discu
ss/tvvzX5tFKA4

Original issue reported on code.google.com by jkoles...@google.com on 10 May 2013 at 1:56

GoogleCodeExporter commented 8 years ago

Original comment by fgalli...@google.com on 17 Jan 2015 at 12:06

GoogleCodeExporter commented 8 years ago

Original comment by fgalli...@google.com on 30 Jan 2015 at 8:33

GoogleCodeExporter commented 8 years ago

Original comment by ya...@google.com on 30 Jan 2015 at 8:50