optimizing xbr scaler

[ksharindam]

Hi @zvezdochiot XBR scaler code uses a precomputed array for RGB to YUV conversion. that occupies 64MB or memory even if the code is not executed. I want to compute rgb to yuv when the code is executed. and will not use that array.

    for (bg = -255; bg < 256; bg++) {
        for (rg = -255; rg < 256; rg++) {
            const uint32_t u = (uint32_t)((-169*rg + 500*bg)/1000) + 128;
            const uint32_t v = (uint32_t)(( 500*rg -  81*bg)/1000) + 128;
            int startg = _max(-bg, _max(-rg, 0));
            int endg = _min(255-bg, _min(255-rg, 255));
            uint32_t y = (uint32_t)(( 299*rg + 1000*startg + 114*bg)/1000);
            c = bg + (rg<<16) + 0x010101 * startg;
            for (g = startg; g <= endg; g++) {
                XBR_RGBtoYUV[c] = ((y++) << 16) + (u << 8) + v;
                c+= 0x010101;
            }
        }
    }

Trying to find a way to convert from r, g and b to rg and bg. by trying to create a reverse equation of c = bg + (rg<<16) + 0x010101 * startg; Please help me.

[zvezdochiot]

Hi @ksharindam .

First I'll try to decompose this into gnumeric. And then we'll see.

https://github.com/ImageProcessing-ElectronicPublications/photoquick-plugins/blob/a2ae6b610f08d86b75742177c6270e12b1a05e7a/src/transform/pixart-scaler/xbr.cpp#L41 https://github.com/ImageProcessing-ElectronicPublications/photoquick-plugins/blob/a2ae6b610f08d86b75742177c6270e12b1a05e7a/src/transform/pixart-scaler/xbr.cpp#L359-L382

---

---

---

---

---

PS: If you multiply the coefficients by 4096 (2**12), then you can multiply with integers, and at the end make the shift >>12.

[ksharindam]

got it. actually rb is simply red-blue, and gb is simply green-blue