johanjohan commented 5 years ago

Since ofPixels.cpp has not been updated for over a year in the master branch, i therefore assume the following issue has not been fixed. i am using of 0.10.1 with win10 vs2017.

I am resizing images/pixels with the following function. That leads to issues:

OF_INTERPOLATE_NEAREST_NEIGHBOR crashes on large sizes
OF_INTERPOLATE_BILINEAR: produces grey only pixels. well yes, "not implemented yet" in ofPixels
OF_INTERPOLATE_BICUBIC produces some singular green pixel/blue artifacts

calling the same function using image.resize() produces best results

question: i assume that i may use pixelsRes = _pixels; to create a full twin copy, right? if yes, then there is the described issue in the pixel resizeTo() function.

CODE:

#include "ofApp.h"

/*
    this function aims to either
        - crop the image to the largest possible dimensions respecting _aspect 
            then scales the longest side to _longestSideLengthPx
        - stretch it respecting _aspect
            then scales the longest side to _longestSideLengthPx
*/

static void resize(
    ofPixels     &_pixels,
    const float  &_longestSideLengthPx,
    const ofVec2f    &_aspect,
    const bool   &_bStretch, // otherwise crop
    const ofInterpolationMethod &_interp    // OF_INTERPOLATE_NEAREST_NEIGHBOR OF_INTERPOLATE_BILINEAR
) {
    ofScaleMode _scaleMode = _bStretch ? OF_SCALEMODE_FILL : OF_SCALEMODE_FIT;
    ofRectangle rectPixels(0, 0, _pixels.getWidth(), _pixels.getHeight());
    ofRectangle rectAspect(0, 0, _aspect.x, _aspect.y);
    rectAspect.scaleTo(rectPixels, _scaleMode);

    ofPixels pixelsRes;
    if (_scaleMode == OF_SCALEMODE_FIT) { // crop
        _pixels.cropTo(pixelsRes, rectAspect.x, rectAspect.y, rectAspect.width, rectAspect.height);
    }
    else { // stretch
        pixelsRes = _pixels;    

        // Q: is this a complete copy with new identical pixels?

        // looks like it does copy all params plus the pixels:
        //template<typename PixelType>
        //void ofPixels_<PixelType>::copyFrom(const ofPixels_<PixelType> & mom) {
        //  if (mom.isAllocated()) {
        //      allocate(mom.getWidth(), mom.getHeight(), mom.getPixelFormat());
        //      memcpy(pixels, mom.getData(), getTotalBytes());
        //  }
        //}
    }

    ofRectangle rectSize;
    if (rectAspect.width > rectAspect.height) {
        assert(rectAspect.width > 0);
        rectSize.set(0, 0, _longestSideLengthPx, _longestSideLengthPx / rectAspect.width * rectAspect.height);
    }
    else { 
        assert(rectAspect.height > 0);
        rectSize.set(0, 0, _longestSideLengthPx / rectAspect.height * rectAspect.width, _longestSideLengthPx);
    }

    ofLogNotice(__FUNCTION__) << "scaling _pixels: rectSize: " << rectSize;
    ofLogNotice(__FUNCTION__) << "_aspect   : " << _aspect;
    ofLogNotice(__FUNCTION__) << "_scaleMode: " << _scaleMode;
    ofLogNotice(__FUNCTION__) << "_interp   : " << _interp;
    ofLogNotice(__FUNCTION__) << "rectPixels: " << rectPixels;
    ofLogNotice(__FUNCTION__) << "rectAspect: " << rectAspect;

#if 1
    // ISSUE is right here

    // OF_INTERPOLATE_NEAREST_NEIGHBOR crashes on large sizes
    // OF_INTERPOLATE_BILINEAR : produces grey only pixels.well yes, "not implemented yet" in ofPixels
    // OF_INTERPOLATE_BICUBIC produces some  singular green pixel / blue artifacts
    // calling above function via image.resize() produces best results
    bool b = pixelsRes.resize(rectSize.width, rectSize.height, _interp); // THIS SEEMS BUGGY  in 0.10.1
    assert(b);
    _pixels = pixelsRes;
#else
    // all ok - done with freeimage functions
    ofImage image(pixelsRes);
    image.resize(rectSize.width, rectSize.height); // freeimage
    _pixels.setFromPixels(
        image.getPixels().getData(),
        image.getWidth(),
        image.getHeight(),
        image.getImageType()
    );
#endif
}
//--------------------------------------------------------------
void ofApp::setup() {

    ofSetLogLevel(OF_LOG_NOTICE);

    ofImage image;
    int longestSide = 15000; // somehow works with small size like 1000, crashes on large sizes
    ofInterpolationMethod interp;

    for (int i = 1; i <= 3; i++)
    {
        ofLogNotice(__FUNCTION__) << i;
        switch (i)
        {
        case 1: interp = OF_INTERPOLATE_NEAREST_NEIGHBOR; break;
        case 2: interp = OF_INTERPOLATE_BILINEAR; break;
        case 3: interp = OF_INTERPOLATE_BICUBIC; break;
        default:
            std::exit(1);
            break;
        }

        bool b = image.load("face.jpg");
        assert(b);

        bool bStretch = true;
        image.load("face.jpg");
        resize(image.getPixels(), longestSide, ofVec2f(16, 9), bStretch, interp);
        // how does the image know that the pixels are scaled now?
        image.setFromPixels(image.getPixels()); // ???
        image.save("face_169_stretch_" + ofToString(i) + ".jpg");

        image.load("face.jpg");
        resize(image.getPixels(), longestSide, ofVec2f(9, 16), bStretch, interp);
        image.setFromPixels(image.getPixels()); // ???
        image.save("face_916_stretch_" + ofToString(i) + ".jpg");

        bStretch = false;
        image.load("face.jpg");
        resize(image.getPixels(), longestSide, ofVec2f(16, 9), bStretch, interp);
        image.setFromPixels(image.getPixels()); // ???
        image.save("face_169_crop_" + ofToString(i) + ".jpg");

        image.load("face.jpg");
        resize(image.getPixels(), longestSide, ofVec2f(9, 16), bStretch, interp);
        image.setFromPixels(image.getPixels()); // ???
        image.save("face_916_crop_" + ofToString(i) + ".jpg");
    }
}

janimatic commented 4 months ago

Hello, Thank you for your great work! I am still experiencing the same crash with ofPixels::resizeTo (OF_INTERPOLATE_NEAREST_NEIGHBOR & OF_INTERPOLATE_BICUBIC, OF_INTERPOLATE_BILINEAR is not implemented). In the meantime, i wrote a very simple method to generate proxies (since most of the time, resizing pixels is only used for performances, otherwise i'd use fbos for full image transformations).

template <typename PixelType>
ofPixels_<PixelType> fastResize(ofPixels_<PixelType> input, int proxyScale = 2)
{
    if (proxyScale < 2 || !(input.isAllocated())) return input;
    int dstWidth = input.getWidth() / proxyScale;
    int dstHeight = input.getHeight() / proxyScale;
    // create image for output  
    ofPixels_<PixelType> output;
    output.allocate(dstWidth, dstHeight, input.getPixelFormat());
    PixelType* dstPixels = output.getData();
    PixelType* srcPixels = input.getData();
    size_t srcWidth = input.getWidth();
    size_t srcHeight = input.getHeight();
    size_t dstIndex = 0;
    for (size_t dsty = 0; dsty < dstHeight; dsty++) {
        for (size_t dstx = 0; dstx < dstWidth; dstx++) {
            size_t pixelIndex = dstIndex * proxyScale;
            dstPixels[dstIndex] = srcPixels[pixelIndex];
            dstIndex++;
        }
    }
    return output;
}

It would be nice to integrate this in the ofPixel class... cheers ! Julien

johanjohan commented 4 months ago

Hey Julien, this was some time ago. I remember that Arturo had fixed some internal indexing vars to size_t (64 bit) at one point, the source of the issue. I cannot recall which OF version though. Probably of 0.10.1 with win10 vs2017. Which version do you work with? Does it appear at all sizes of images or above a 32bit threshold? Cheers M

janimatic commented 4 months ago

Hello Michael, I am using the latest master (cmake based project). The crash happened only with ofPixels_<float>, ofPixels_<unsigned char> worked fine if i remember well. I suspect there might eventually be something wrong in pointer's arithmetics and looping the colors using getBytesPerPixel() as limit, but i'd need to look at it deeper when i have time. Thanks for your feedback, i'd keep you informed with more testings asap...

dimitre commented 4 months ago

Hello @janimatic can you please write the minimal code example that can reproduce the issue and post here? Thank you

janimatic commented 4 months ago

Hello @dimitre @johanjohan Here is a minimal code example... thanks!

#include "ofMain.h"
template <typename PixelType>
void resizeTest(int width, int height, int thumbnailProxy, ofInterpolationMethod interp) {
    ofPixels_<PixelType> pixels;
    pixels.allocate(width, height, OF_IMAGE_COLOR_ALPHA);
    auto proxy = pixels;
    proxy.resize(pixels.getWidth() / thumbnailProxy, pixels.getHeight() / thumbnailProxy, interp);
    ofLogWarning() << proxy.getColor(0);
}

int main( ){
    int thumbnailProxy = 16;
    resizeTest<unsigned char>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_NEAREST_NEIGHBOR);
    resizeTest<unsigned char>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_BILINEAR);
    resizeTest<unsigned char>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_BICUBIC);
    resizeTest<float>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_NEAREST_NEIGHBOR);
    resizeTest<float>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_BILINEAR);
    resizeTest<float>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_BICUBIC);
}

janimatic commented 4 months ago

Hey @dimitre @johanjohan , I think replacing line https://github.com/openframeworks/openFrameworks/blob/7a09e9eee85b8b98f8447072d6259544a0b9cb37/libs/openFrameworks/graphics/ofPixels.cpp#L1347 size_t bytesPerPixel = getBytesPerPixel(); by size_t bytesPerPixel = channelsFromPixelFormat(pixelFormat); would fix OF_INTERPOLATE_NEAREST_NEIGHBOR (renaming the variable accordingly of course) sizeof(int) was ok but sizeof(float) 4 would make it fail. I don't really understand why the pointer would use the size of pixel type to walk through color channels... I didn't check for OF_INTERPOLATE_BICUBIC though ! thanks

janimatic commented 4 months ago

PS : This seem to work (but should be tested)

//----------------------------------------------------------------------
template<typename PixelType>
bool ofPixels_<PixelType>::resizeTo(ofPixels_<PixelType>& dst, ofInterpolationMethod interpMethod) const {
    if (&dst == this) {
        return true;
    }

    if (!(isAllocated()) || !(dst.isAllocated()) || getBytesPerPixel() != dst.getBytesPerPixel()) return false;

    size_t srcWidth = getWidth();
    size_t srcHeight = getHeight();
    size_t dstWidth = dst.getWidth();
    size_t dstHeight = dst.getHeight();
    size_t bytesPerPixel = getBytesPerPixel();
    size_t channels = channelsFromPixelFormat(pixelFormat);

    PixelType* dstPixels = dst.getData();

    switch (interpMethod) {

        //----------------------------------------
    case OF_INTERPOLATE_NEAREST_NEIGHBOR: {
        size_t dstIndex = 0;
        float srcxFactor = (float)srcWidth / dstWidth;
        float srcyFactor = (float)srcHeight / dstHeight;
        float srcy = 0.5;
        for (size_t dsty = 0; dsty < dstHeight; dsty++) {
            float srcx = 0.5;
            size_t srcIndex = static_cast<size_t>(srcy) * srcWidth;
            for (size_t dstx = 0; dstx < dstWidth; dstx++) {
                size_t pixelIndex = static_cast<size_t>(srcIndex + srcx) * channels;
                for (size_t k = 0; k < channels; k++) {
                    dstPixels[dstIndex] = pixels[pixelIndex];
                    dstIndex++;
                    pixelIndex++;
                }
                srcx += srcxFactor;
            }
            srcy += srcyFactor;
        }
    }break;

        //----------------------------------------
    case OF_INTERPOLATE_BILINEAR:
        // not implemented yet
        ofLogError("ofPixels") << "resizeTo(): bilinear resize not implemented, not resizing";
        break;

        //----------------------------------------
    case OF_INTERPOLATE_BICUBIC:
        float px1, py1;
        float px2, py2;
        float px3, py3;

        float srcColor = 0;
        float interpCol;
        size_t patchRow;
        size_t patchIndex;
        float patch[16];

        size_t srcRowBytes = srcWidth * bytesPerPixel;
        size_t loIndex = (srcRowBytes)+1;
        //size_t hiIndex = (srcWidth * srcHeight * bytesPerPixel) - (srcRowBytes)-1;
        size_t hiIndex = (srcWidth * srcHeight * channels) - (srcRowBytes)-1;

        for (size_t dsty = 0; dsty < dstHeight; dsty++) {
            for (size_t dstx = 0; dstx < dstWidth; dstx++) {

                //size_t   dstIndex0 = (dsty * dstWidth + dstx) * bytesPerPixel;
                size_t   dstIndex0 = (dsty * dstWidth + dstx) * channels;
                float srcxf = srcWidth * (float)dstx / (float)dstWidth;
                float srcyf = srcHeight * (float)dsty / (float)dstHeight;
                size_t   srcx = static_cast<size_t>(std::min(srcWidth - 1, static_cast<size_t>(srcxf)));
                size_t   srcy = static_cast<size_t>(std::min(srcHeight - 1, static_cast<size_t>(srcyf)));
                //size_t   srcIndex0 = (srcy * srcWidth + srcx) * bytesPerPixel;
                size_t   srcIndex0 = (srcy * srcWidth + srcx) * channels;

                px1 = srcxf - srcx;
                py1 = srcyf - srcy;
                px2 = px1 * px1;
                px3 = px2 * px1;
                py2 = py1 * py1;
                py3 = py2 * py1;

                //for (size_t k = 0; k < bytesPerPixel; k++) {
                for (size_t k = 0; k < channels; k++) {
                    size_t   dstIndex = dstIndex0 + k;
                    size_t   srcIndex = srcIndex0 + k;

                    for (size_t dy = 0; dy < 4; dy++) {
                        patchRow = srcIndex + ((dy - 1) * srcRowBytes);
                        for (size_t dx = 0; dx < 4; dx++) {
                            //patchIndex = patchRow + (dx - 1) * bytesPerPixel;
                            patchIndex = patchRow + (dx - 1) * channels;
                            if ((patchIndex >= loIndex) && (patchIndex < hiIndex)) {
                                srcColor = pixels[patchIndex];
                            }
                            patch[dx * 4 + dy] = srcColor;
                        }
                    }

                    interpCol = (PixelType)bicubicInterpolate(patch, px1, py1, px2, py2, px3, py3);
                    dstPixels[dstIndex] = interpCol;
                }

            }
        }
        break;
    }

    return true;
}

But with this test

#include "ofMain.h"
//========================================================================

template <typename PixelType>
void resizeTest(int width, int height, int thumbnailProxy, ofInterpolationMethod interp, ofColor col) {
    ofPixels_<PixelType> pixels;
    pixels.allocate(width, height, OF_IMAGE_COLOR_ALPHA);
    pixels.setColor(col);
    ofLogWarning() << pixels.getColor(0);
    auto proxy = pixels;
    proxy.resize(pixels.getWidth() / thumbnailProxy, pixels.getHeight() / thumbnailProxy, interp);
    ofLogWarning() << proxy.getColor(0);
}

int main() {
    int thumbnailProxy = 16;
    resizeTest<unsigned char>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_NEAREST_NEIGHBOR, ofColor(255, 255, 255, 255));
    resizeTest<unsigned char>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_BICUBIC, ofColor(255, 255, 255, 255));
    resizeTest<float>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_NEAREST_NEIGHBOR, ofColor(1, 1, 1, 1));
    resizeTest<float>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_BICUBIC, ofColor(1, 1, 1, 1));
}

i am facing strange logged values from a simple pixels.setColor(ofColor(1, 1, 1, 1)); on float pixels :

[warning] 255, 255, 255, 255
[warning] 255, 255, 255, 255
[warning] 255, 255, 255, 255
[warning] 0, 255, 255, 255
[warning] 0.00392157, 0.00392157, 0.00392157, 0.00392157
[warning] 0.00392157, 0.00392157, 0.00392157, 0.00392157
[warning] 0.00392157, 0.00392157, 0.00392157, 0.00392157
[warning] 0, 0, 0, 0

dimitre commented 4 months ago

I'm proposing a simplification of channel / bytes handling here:

https://github.com/openframeworks/openFrameworks/pull/7929

dimitre commented 4 months ago

Please test the PR to see if it works for you @janimatic there are lots of changes but the main one is this:

    pixelsSize = newSize / sizeof(PixelType);

and

    pixelsSize = bytesFromPixelFormat(w,h,_pixelFormat) / sizeof(PixelType);

we should not divide the pixelsSize by the sizeof(PixelType), it works with char because it is / 1 (memory size of char) but it will cause problems with float etc, cc: @oftheo @NickHardeman

janimatic commented 4 months ago

Hello @dimitre , thank you very much for this. For the moment, testing with float pixels,

if i use this minimal test, i get funky logged values


#include "ofMain.h"
//========================================================================

template void resizeTest(int width, int height, int thumbnailProxy, ofInterpolationMethod interp, ofColor col) { ofPixels_ pixels; pixels.allocate(width, height, OF_IMAGE_COLOR_ALPHA); pixels.setColor(col); ofLogWarning() << pixels.getColor(0); ofLogWarning() << pixels.getColor(0).r << "," << pixels.getColor(0).g << "," << pixels.getColor(0).b << "," << pixels.getColor(0).a; auto proxy = pixels; proxy.resize(pixels.getWidth() / thumbnailProxy, pixels.getHeight() / thumbnailProxy, interp); ofLogWarning() << proxy.getColor(0); ofLogWarning() << proxy.getColor(0).r << "," << proxy.getColor(0).g << "," << proxy.getColor(0).b << "," << proxy.getColor(0).a; }

int main() { int thumbnailProxy = 16; resizeTest(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_NEAREST_NEIGHBOR, ofColor(255, 255, 255, 255)); resizeTest(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_BICUBIC, ofColor(255, 255, 255, 255)); resizeTest(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_NEAREST_NEIGHBOR, ofColor(1, 1, 1, 1)); resizeTest(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_BICUBIC, ofColor(1, 1, 1, 1)); } / outputs [warning] 255, 255, 255, 255 [warning] ÿ,ÿ,ÿ,ÿ [warning] 80, 1, 7, 76 [warning] P,,,L [warning] 255, 255, 255, 255 [warning] ÿ,ÿ,ÿ,ÿ [warning] 0, 255, 255, 255 [warning] [warning] 0.00392157, 0.00392157, 0.00392157, 0.00392157 [warning] 0.00392157,0.00392157,0.00392157,0.00392157 [warning] 3.53908e+07, 6.0396e-43, 3.55085e+07, 6.0396e-43 [warning] 3.53908e+07,6.0396e-43,3.55085e+07,6.0396e-43 [warning] 0.00392157, 0.00392157, 0.00392157, 0.00392157 [warning] 0.00392157,0.00392157,0.00392157,0.00392157 [warning] 0, 0, 0, 0 [warning] 0,0,0,0 /

Is my testing method incorrect?
I guess i shouldn't complicate the testing with Qt but ...just for info
* If i try tu use Qt6 to convert to QImage::Format_RGBA32FPx4 it doesn't output a QImage,

QTableWidgetItem item = new QTableWidgetItem(text); QImage image((const uchar )pixels.getData(), pixels.getWidth(), pixels.getHeight(), QImage::Format_RGBA32FPx4); item->setData(Qt::DecorationRole, QPixmap::fromImage(image)); item->setSizeHint(QSize(pixels.getWidth(), pixels.getHeight()));

* If I use Qt5 compatible code and a silly pixel by pixel conversion (float to int)
Qt print QImage::setPixelColor: color is invalid

                QImage image(pixels.getWidth(), pixels.getHeight(), QImage::Format_ARGB32);
                for (int y = 0; y < image.height(); y++) {
                    for (int x = 0; x < image.width(); x++) {
                        ofColor col = pixels.getColor(x, y);
                        image.setPixelColor(x, y, QColor(col.r * 255, col.g * 255, col.b * 255, col.a * 255));
                    }
                }
                item->setData(Qt::DecorationRole, QPixmap::fromImage(image));

I can create a tiny exr app quickly if that helps for a more concrete and visual testing.
Tell me how i can help.

PS : I once wrote a template class to avoid dealing with pixels pointers arithmetic wich drive me nut when writing openfx cpu image effect plugins, without any allocation unless (requested). It was used with float image (in fusion) and arbitrary data type.
Not sure if that usefull, it's not the best, but i post it here just in case ?

Thank you very much for your great work !

///

/// An Image Class making color access easier /// Using internal image data pointer, no copy occurs. /// It just makes indexing easier /// But we cannot use pointer arithmetic on it, since the data is still unidimensional (such as float* RGBARGBARGBA....) ///

/// any number type, openfx in resolve typically uses float images: auto srcColors = ColorImage(src);

// USE_VEC seams to be broken now ? It would be nice to get rid of raw pointers though.... //#define USE_VEC // vector is handy for debugging, but slightly slower : // using CHANNELTYPE* _array, raw pointer makes no copy... template class ColorImage { private:

ifdef USE_VEC

std::vector<CHANNELTYPE> _array;

else

CHANNELTYPE* _array;

endif // USE_VEC

int _w, _h, _nComponents;
u64 _index = 0;     // keep track of internal index : this is not realiable we user uses 
int _channel = 0; // to interpret single channel, set this to desired channel index and just use pixels(x,y) method as usual...
int _stride;
bool internalArray = false;

public: ColorImage() {} // openfx image constructor ColorImage(OFX::Image* img) : _w(img->getBounds().x2 - img->getBounds().x1), _h(img->getBounds().y2 - img->getBounds().y1), _stride(img->getRowBytes()), _nComponents(img->getPixelComponentCount()) { // TODO : make a true monochromatic copy in the constructor // https://www.cs.odu.edu/~zeil/cs330/latest/Public/big3/index.html

ifdef USE_VEC

    auto ptr = static_cast<CHANNELTYPE*>(img->getPixelData());
    u64 arraySize = _w * _h * _nComponents;
    // fill the vector with OFX::Image data (float* pointer)
    _array.insert(_array.end(), ptr, ptr + arraySize);

else

    _array = static_cast<CHANNELTYPE*>(img->getPixelData());

endif // USE_VEC

}
// any imaga data pointer constructor
ColorImage(CHANNELTYPE* ptr, int width, int height, int nComponents) {
    _array = ptr;
    _w = width;
    _h = height;
    _nComponents = nComponents;
    _calculateStride();
}
// copy constructor
//ColorImage(ColorImage& other) {
ColorImage(const ColorImage& other) {
    ColorImage tmp(other._array, other._w, other._h, other._nComponents);
}
ColorImage& operator=(const ColorImage& other)
{
    this->_array = other._array;
    this->_w = other._w;
    this->_h = other._h;
    this->_nComponents = other._nComponents;
    return *this;
}
// initial allocation : provide the dimension, and the array of any type will be filled automatically...
ColorImage(int width, int height, int nComponents) {
    _w = width;
    _h = height;
    _nComponents = nComponents;
    internalArray = true;
    u64 arraySize = _w * _h * _nComponents;

ifdef USE_VEC

    // fill the vector with custom data (any* pointer)
    _array.resize(arraySize, CHANNELTYPE());

else

    _array = new CHANNELTYPE[arraySize]();
    // Notice the parenthesis in new CHANNELTYPE[arraySize]() : 
    // Since CHANNELTYPE's constructor is called, there is no need to set initial data manually in a loop...

endif // USE_VEC

    _calculateStride();
}
void copyPixelFrom(u64 idx, ColorImage<CHANNELTYPE>& from) {
    _array[_nComponents * idx + 0] = from._array[_nComponents * idx + 0];
    _array[_nComponents * idx + 1] = from._array[_nComponents * idx + 1];
    _array[_nComponents * idx + 2] = from._array[_nComponents * idx + 2];
    _array[_nComponents * idx + 3] = from._array[_nComponents * idx + 3];
}
void copyPixelFrom(int x, int y, ColorImage<CHANNELTYPE>& from) {
    _array[_nComponents * pixelIndex(_w, _h, x, y) + 0] = from._array[_nComponents * pixelIndex(_w, _h, x, y) + 0];
    _array[_nComponents * pixelIndex(_w, _h, x, y) + 1] = from._array[_nComponents * pixelIndex(_w, _h, x, y) + 1];
    _array[_nComponents * pixelIndex(_w, _h, x, y) + 2] = from._array[_nComponents * pixelIndex(_w, _h, x, y) + 2];
    _array[_nComponents * pixelIndex(_w, _h, x, y) + 3] = from._array[_nComponents * pixelIndex(_w, _h, x, y) + 3];
}
void _calculateStride() {
    // we don't need stride in bytes : the float* pointer addresses elements, not bytes...
    // see https://medium.com/@oleg.shipitko/what-does-stride-mean-in-image-processing-bba158a72bcd
    // see https://cplusplus.com/forum/windows/162811/
    // we keep this here just for reference.
    // This result seems to be conform with OFX::Image getRowBytes() result anyway...
    _stride = _w * sizeof(CHANNELTYPE) * _nComponents;
    // No padding ? Not sure of this...
    //_stride = ((_stride + 63) / 64) * 64;
}
int getNumberOfChannels() {
    return _nComponents;
}
int bytesPerRow() {
    return _stride;
}
~ColorImage() {
    if (internalArray) {

ifdef USE_VEC

        _array.clear();

else

        // rewind before delete ?
        // https://stackoverflow.com/questions/41184086/why-does-my-program-crash-when-i-increment-a-pointer-and-then-delete-it
        _array = 0;
        delete[] _array;

endif // USE_VEC

    }
    //if (internalArray) free(_array);
}
bool isAllocated() {
    return _array != nullptr;
    //return _array != nullptr && _channel > 0;
}
void release() {

ifdef USE_VEC

    _array.clear();

else

    _array = 0;
    delete[] _array;

endif // USE_VEC

}
// reset to default value
void reset(){
    u64 arraySize = _w * _h * _nComponents;
    for (auto i=0; i<arraySize; i++)
        _array[i] = CHANNELTYPE();
}
int getWidth() { return _w; }
int getHeight() { return _h; }

// toonz compatibility
int getLx() { return _w; }
int getLy() { return _h; }
int getWrap() { return _w; }
// to use for channel access 
void setChannel(int c) {
    // for now, we just pass an int index to define the channel
    // TODO:
    // a bitset https://en.cppreference.com/w/cpp/utility/bitset or 4 booleans or one string ?
    // make sure it's very efficient... : it's just an offset: r:0 g:1 b:2 a:4
    assert(c <= _nComponents && c >= 0);
    if (c > _nComponents) c = _nComponents;
    if (c < 0) c = 0;
    _channel = c;
}
int channel() { return _channel; }
// returns a pointer address with offset
CHANNELTYPE* pixels(u64 idx) {
    assert(idx < _w * _h && idx >= 0);
    if (idx > _w * _h) idx = _w * _h;
    if (idx < 0 ) idx = 0;
    // https://stackoverflow.com/questions/6485496/how-to-get-stdvector-pointer-to-the-raw-data
    return &_array[_nComponents * idx + _channel];
    //return _array + (_nComponents * idx + 0);
}
// returns a pointer address with offset
// global pointer offset is equivalent to red offset (first index of the pixel)

ifdef USE_VEC

//  Erro C2440  'return' : impossible de convertir de 'const _Ty *' en 'CHANNELTYPE *'
CHANNELTYPE* pixels(int x, int y)  {

else

CHANNELTYPE* pixels(int x, int y) const {

endif // USE_VEC

    // clamp x & y, if necessary
    if (x >= _w) x = _w - 1;
    if (x < 0) x = 0;
    if (y >= _h) y = _h - 1;
    if (y < 0) y = 0;
    // https://stackoverflow.com/questions/6485496/how-to-get-stdvector-pointer-to-the-raw-data
    return &_array[_nComponents * pixelIndex(_w, _h, x, y) + _channel]; // _channel : monochromatic access...
}
bool areEqual(u64 idxA, u64 idxB, CHANNELTYPE threshold) const {
    auto a = pixel(idxA);
    auto b = pixel(idxB);
    return std::max({ abs((CHANNELTYPE)a.r - b.r), abs((CHANNELTYPE)a.g - b.g),
                     abs((CHANNELTYPE)a.b - b.b), abs((CHANNELTYPE)a.a - b.a) }) < threshold;
}

CHANNELTYPE* getPreviousPixel(CHANNELTYPE* ptr) {
    return ptr - _nComponents;
}
CHANNELTYPE* getNextPixel(CHANNELTYPE* ptr) {
    return ptr + _nComponents;
}
CHANNELTYPE* getPreviousRow(CHANNELTYPE* ptr) {
    // we don't need stride in bytes : the float* pointer addresses elements, not bytes...
    // return reinterpret_cast<CHANNELTYPE*>(reinterpret_cast<std::uint8_t*>(ptr) - _stride);
    return ptr - (_w * _nComponents);
}
CHANNELTYPE* getNextRow(CHANNELTYPE* ptr) {
    // return reinterpret_cast<CHANNELTYPE*>(reinterpret_cast<std::uint8_t*>(ptr) + _stride);
    return ptr + (_w * _nComponents);
}

// unidimensional pixel indexing (i)
// Safe access to pixel's components (rgba) :
// It returns 0 if the components doesn't exit
CHANNELTYPE r(u64 idx) {
    //assert(_nComponents > 0);
    if (_nComponents > 0)
        return _array[_nComponents * idx + 0];
    return 0.0;
}
CHANNELTYPE g(u64 idx) {
    //assert(_nComponents > 1);
    if (_nComponents > 1)
        return _array[_nComponents * idx + 1];
    return 0.0;
}
CHANNELTYPE b(u64 idx) {
    //assert(_nComponents > 2);
    if (_nComponents > 2)
        return _array[_nComponents * idx + 2];
    return 0.0;
}
CHANNELTYPE a(u64 idx) {
    //assert(_nComponents > 3);
    if (_nComponents > 3)
        return _array[_nComponents * idx + 3];
    return 0.0;
}
RgbaPixel<CHANNELTYPE> pixel(u64 idx) {
    return RgbaPixel<CHANNELTYPE> {r(idx), g(idx), b(idx), a(idx)};
}
// bidimensional pixel indexing (x,y)
// Safe access to pixel's components (rgba) :
// It returns 0 if the components doesn't exit
CHANNELTYPE r(int x, int y) {
    return r(pixelIndex(_w, _h, x, y));
}
CHANNELTYPE g(int x, int y) {
    return g(pixelIndex(_w, _h, x, y));
}
CHANNELTYPE b(int x, int y) {
    return b(pixelIndex(_w, _h, x, y));
}
CHANNELTYPE a(int x, int y) {
    return a(pixelIndex(_w, _h, x, y));
}
RgbaPixel<CHANNELTYPE> pixel(int x, int y) {
    return pixel(pixelIndex(_w, _h, x, y));
}
// unidimensional pixel indexing (i)
// Safe pixel's components setter (rgba) :
// Skipping if the components doesn't exit
void setR(CHANNELTYPE v, u64 idx) {
    //assert(_nComponents > 0);
    if (_nComponents > 0)
        _array[_nComponents * idx + 0] = v;
}
void setG(CHANNELTYPE v, u64 idx) {
    //assert(_nComponents > 1);
    if (_nComponents > 1)
        _array[_nComponents * idx + 1] = v;
}
void setB(CHANNELTYPE v, u64 idx) {
    //assert(_nComponents > 2);
    if (_nComponents > 2)
        _array[_nComponents * idx + 2] = v;
}
void setA(CHANNELTYPE v, u64 idx) {
    //assert(_nComponents > 3);
    if (_nComponents > 3)
        _array[_nComponents * idx + 3] = v;
}
void setPixel(CHANNELTYPE r, CHANNELTYPE g, CHANNELTYPE b, CHANNELTYPE a, u64 idx) {
    setR(r, idx);
    setG(g, idx);
    setB(b, idx);
    setA(a, idx);
}
void setPixel(RgbaPixel<CHANNELTYPE> pixel, u64 idx) {
    setPixel(pixel.r, pixel.g, pixel.b, pixel.a, idx);
}
// bidimensional pixel indexing (x,y)
// Safe pixel's components setter (rgba) :
// Skiping if the components doesn't exit
void setR(CHANNELTYPE v, int x, int y) {
    setR(v, pixelIndex(_w, _h, x, y));
}
void setG(CHANNELTYPE v, int x, int y) {
    setG(v, pixelIndex(_w, _h, x, y));
}
void setB(CHANNELTYPE v, int x, int y) {
    setB(v, pixelIndex(_w, _h, x, y));
}
void setA(CHANNELTYPE v, int x, int y) {
    setA(v, pixelIndex(_w, _h, x, y));
}
void setPixel(CHANNELTYPE r, CHANNELTYPE g, CHANNELTYPE b, CHANNELTYPE a, int x, int y) {
    setPixel(r, g, b, a, pixelIndex(_w, _h, x, y));
}
void setPixel(RgbaPixel<CHANNELTYPE> pixel, int x, int y) {
    setPixel(pixel, pixelIndex(_w, _h, x, y));
}
// unidimensional pixel indexing (i)
CHANNELTYPE* operator[](u64 index) const
{
    return &this->_array[_nComponents * index];
}
// raw data getter
CHANNELTYPE* data()
{

ifdef USE_VEC

    return _array.data();

else

    return _array;

endif // USE_VEC

}
// raw data setter
void setData(CHANNELTYPE* data)
{
    _array = data;
}

};

dimitre commented 4 months ago

I didnt' read everything yet, but some suggestions in your code: 1 - if you convert to int before log it won't display chars. ofLogWarning() << int(pixels.getColor(0).r) << "," << int(pixels.getColor(0).g) << "," << int(pixels.getColor(0).b) << "," << int(pixels.getColor(0).a);

2 - ofColor(1, 1, 1, 1) is almost black. maybe you wanted ofFloatColor(1, 1, 1, 1)

dimitre commented 4 months ago

And I've changed the NEAREST_NEIGHBOR code, but didn't test it. if you find it is wrong you can revert to OF core one.

dimitre commented 4 months ago

it seems to be working here :

#include "ofMain.h"
//========================================================================

template <typename PixelType>
void resizeTest(int width, int height, int thumbnailProxy, ofInterpolationMethod interp, ofColor col) {
    ofPixels_<PixelType> pixels;
    pixels.allocate(width, height, OF_IMAGE_COLOR_ALPHA);
    pixels.setColor(col);
    {
        ofColor color = pixels.getColor(pixels.getWidth() * .5, pixels.getHeight() * .5);
        ofLogWarning() << color;
    }
//  ofLogWarning() << pixels.getColor(0);
//  ofLogWarning() << int(pixels.getColor(0).r) << "," << int(pixels.getColor(0).g) << "," << int(pixels.getColor(0).b) << "," << int(pixels.getColor(0).a);
    auto proxy = pixels;
    proxy.resize(pixels.getWidth() / thumbnailProxy, pixels.getHeight() / thumbnailProxy, interp);
    {
        ofColor color = pixels.getColor(pixels.getWidth() * .5, pixels.getHeight() * .5);
        ofLogWarning() << color;
    }
//  ofLogWarning() << color;
//  ofLogWarning() << int(proxy.getColor(0).r) << "," << int(proxy.getColor(0).g) << "," << int(proxy.getColor(0).b) << "," << int(proxy.getColor(0).a);
}

int main() {
    int thumbnailProxy = 16;
    cout << "--- one" << endl;
    resizeTest<unsigned char>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_NEAREST_NEIGHBOR, ofColor(255, 255, 0, 255));
    cout << "--- two" << endl;
    resizeTest<unsigned char>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_BICUBIC, ofColor(255, 255, 0, 255));
    cout << "--- three" << endl;
    resizeTest<float>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_NEAREST_NEIGHBOR, ofFloatColor(1, 1, 0, 1));
    cout << "--- four" << endl;
    resizeTest<float>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_BICUBIC, ofFloatColor(1, 1, 0, 1));
}

janimatic commented 4 months ago

@dimitre great! i just modified it with templated color and it seem to work with floats...perfect...

template <typename PixelType>
void resizeTest(int width, int height, int thumbnailProxy, ofInterpolationMethod interp, ofColor_<PixelType> col) {
    ofPixels_<PixelType> pixels;
    pixels.allocate(width, height, OF_IMAGE_COLOR_ALPHA);
    pixels.setColor(col);
    {
        ofColor_<PixelType> color = pixels.getColor(pixels.getWidth() * .5, pixels.getHeight() * .5);
        ofLogWarning() << color;
    }
    auto proxy = pixels;
    proxy.resize(pixels.getWidth() / thumbnailProxy, pixels.getHeight() / thumbnailProxy, interp);
    {
        ofColor_<PixelType> color = pixels.getColor(pixels.getWidth() * .5, pixels.getHeight() * .5);
        ofLogWarning() << color;
    }
}
/*
[warning] 255, 255, 0, 255
[warning] 255, 255, 0, 255
[warning] 255, 255, 0, 255
[warning] 255, 255, 0, 255
[warning] 1, 1, 0, 1
[warning] 1, 1, 0, 1
[warning] 1, 1, 0, 1
[warning] 1, 1, 0, 1
*/

janimatic commented 4 months ago

@dimitre for the moment, the resize seem broken with 8 bit pixels too (just tested in the context of Qt thumbnail generation that works in master, I didn't write a simple test case with picture. Hopefully i'll do when i get time...) thanks

dimitre commented 4 months ago

Thanks, let me know when you have the tests. you can try a minimal change to the core by finding "pixelsSize = " and removing sizeof()

janimatic commented 4 months ago

hello @dimitre Your small fix suggestion on master seems perfect. For the refactor pr testing, I wrote some test code for float images io... It could not be very minimal though, since minimal float image io is not that small... Here is a test using tinyexr and stbimage libs + ofImage (which internally resizes using freeimage, and can read/write 32 bit tifs) I've attached the full source + image data 007_imageio.zip Thanks !

#include "ofMain.h"
//========================================================================
#include "imageIO.h"

template <typename PixelType>
void resizeTest(int width, int height, int thumbnailProxy, ofInterpolationMethod interp, ofColor_<PixelType> col) {
    ofPixels_<PixelType> pixels;
    pixels.allocate(width, height, OF_IMAGE_COLOR_ALPHA);
    pixels.setColor(col);
    {
        ofColor_<PixelType> color = pixels.getColor(pixels.getWidth() * .5, pixels.getHeight() * .5);
        ofLogNotice() << color;
    }
    auto proxy = pixels;
    proxy.resize(pixels.getWidth() / thumbnailProxy, pixels.getHeight() / thumbnailProxy, interp);
    {
        ofColor_<PixelType> color = pixels.getColor(pixels.getWidth() * .5, pixels.getHeight() * .5);
        ofLogNotice() << color;
    }
}

template <typename PixelType>
void resizeIOTest_ofImage(int thumbnailProxy, string prefix, string extension) {
    ofLogNotice() << "ofImage_<float> test (using freeimage resize)";
    ofImage_<PixelType> floatimage;
    auto filename = prefix + "." + extension;
    if (!floatimage.load(ofToDataPath(filename, true)))
        ofLogWarning() << "failed while loading ofImage (using freeimage) " << filename;
    else
        ofLogNotice() << "success while loading ofImage (using freeimage) " << filename;
    filename = prefix + "_copy." + extension;
    if (!floatimage.save(ofToDataPath(filename, true)))
        ofLogWarning() << "failed while saving ofImage (using freeimage) " << filename;
    else
        ofLogNotice() << "success while saving ofImage (using freeimage) " << filename;
    floatimage.resize(floatimage.getWidth() / thumbnailProxy, floatimage.getHeight() / thumbnailProxy);
    filename = prefix + "_proxy." + extension;
    if (!floatimage.save(ofToDataPath(filename, true)))
        ofLogWarning() << "failed while saving ofImage (using freeimage) " << filename;
    else
        ofLogNotice() << "success while saving ofImage (using freeimage) " << filename;
}

template <typename PixelType>
void resizeIOTest_ofPixels(int thumbnailProxy, string prefix, string extension, ofInterpolationMethod interp) {
    ofLogNotice() << "ofPixels test using tinyexr & stb to load and save float images and proxies";
    int width, height, channels;
    auto filename = prefix + "." + extension;
    PixelType* data = ImageIO::loadImage(ofToDataPath(filename, true), width, height, channels);
    ofPixels_<PixelType> pixels;
    pixels.setFromPixels(data, width, height, OF_IMAGE_COLOR_ALPHA);
    filename = prefix + "_copy." + extension;
    ImageIO::saveImage(ofToDataPath(filename, true), width, height, channels, pixels.getData());
    pixels.resize(pixels.getWidth() / thumbnailProxy, pixels.getHeight() / thumbnailProxy, interp);
    width = pixels.getWidth() / thumbnailProxy;
    height = pixels.getHeight() / thumbnailProxy;
    filename = prefix + "_proxy." + extension;
    ImageIO::saveImage(ofToDataPath(filename, true), width, height, channels, pixels.getData());
}

int main() {
    int thumbnailProxy = 16;
    ofLogNotice();
    ofLogNotice() << "--- one";
    resizeTest<unsigned char>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_NEAREST_NEIGHBOR, ofColor(255, 255, 0, 255));
    ofLogNotice();
    ofLogNotice() << "--- two";
    resizeTest<unsigned char>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_BICUBIC, ofColor(255, 255, 0, 255));
    ofLogNotice();
    ofLogNotice() << "--- three";
    resizeTest<float>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_NEAREST_NEIGHBOR, ofFloatColor(1, 1, 0, 1));
    ofLogNotice();
    ofLogNotice() << "--- four";
    resizeTest<float>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_BICUBIC, ofFloatColor(1, 1, 0, 1));

    ofLogNotice();
    ofLogNotice() << "convert pixels int to float with ofPixels =";
    ofPixels_<unsigned char> pixels;
    pixels.allocate(1920, 1080, OF_IMAGE_COLOR_ALPHA);
    pixels.setColor(ofColor(255, 255, 0, 255));
    {
        ofColor_<unsigned char> color = pixels.getColor(pixels.getWidth() * .5, pixels.getHeight() * .5);
        ofLogNotice() << color;
    }
    ofPixels_<float> floatpixels;
    floatpixels = pixels;
    auto color = floatpixels.getColor(pixels.getWidth() * .5, pixels.getHeight() * .5);
    ofLogNotice() << color;

    ofLogNotice();
    ofLogNotice() << "--- 32 bits linear tif resizeIOTest_ofImage";
    resizeIOTest_ofImage<float>(thumbnailProxy, "Digital_LAD_HD720", "tif");
    ofLogNotice();
    ofLogNotice() << "--- 32 bits linear exr resizeIOTest_ofPixels OF_INTERPOLATE_NEAREST_NEIGHBOR";
    resizeIOTest_ofPixels<float>(thumbnailProxy, "Digital_LAD_HD720", "exr", ofInterpolationMethod::OF_INTERPOLATE_NEAREST_NEIGHBOR);
    ofLogNotice();
    ofLogNotice() << "--- 32 bits linear exr resizeIOTest_ofPixels OF_INTERPOLATE_BICUBIC";
    resizeIOTest_ofPixels<float>(thumbnailProxy, "Digital_LAD_HD720", "exr", ofInterpolationMethod::OF_INTERPOLATE_BICUBIC);

}

dimitre commented 4 months ago

@janimatic can you please test this other PR? https://github.com/openframeworks/openFrameworks/pull/7936

janimatic commented 4 months ago

@dimitre hello!

the Qt thumbs are working with int anf float pixels (Qt6 QImage::Format_RGBA32FPx4 thumb looks ultra pixalated.)
ofImage_ / freeimage resize removes the alpha channel
this test code generates a pixelated exr (compared to the tif generated by ofImage_ / freeimage resize) I am not sure if that's my code of the ofPixel resize that is broken (looks like a wrong channel offset). But since ofpixels to Qt6 thumb looks pixelated too i guess it's of resize that's still broken. (see ofimage_resize_tif.png vs ofpixel_resize_exr.png) (Note : my previous test code was plain wrong : it divide by thumbnailProxy twice)

#include "ofMain.h"
//========================================================================
#include "imageIO.h"

template <typename PixelType>
void resizeTest(int width, int height, int thumbnailProxy, ofInterpolationMethod interp, ofColor_<PixelType> col) {
    ofPixels_<PixelType> pixels;
    pixels.allocate(width, height, OF_IMAGE_COLOR_ALPHA);
    pixels.setColor(col);
    {
        ofColor_<PixelType> color = pixels.getColor(pixels.getWidth() * .5, pixels.getHeight() * .5);
        ofLogNotice() << color;
    }
    auto proxy = pixels;
    proxy.resize(pixels.getWidth() / thumbnailProxy, pixels.getHeight() / thumbnailProxy, interp);
    {
        ofColor_<PixelType> color = pixels.getColor(pixels.getWidth() * .5, pixels.getHeight() * .5);
        ofLogNotice() << color;
    }
}

template <typename PixelType>
void resizeIOTest_ofImage(int thumbnailProxy, string prefix, string extension) {
    ofLogNotice() << "ofImage_<float> test (using freeimage resize)";
    ofImage_<PixelType> floatimage;
    auto filename = prefix + "." + extension;
    if (!floatimage.load(ofToDataPath(filename, true)))
        ofLogWarning() << "failed while loading ofImage (using freeimage) " << filename;
    else
        ofLogNotice() << "success while loading ofImage (using freeimage) " << filename;
    filename = prefix + "_copy." + extension;
    if (!floatimage.save(ofToDataPath(filename, true)))
        ofLogWarning() << "failed while saving ofImage (using freeimage) " << filename;
    else
        ofLogNotice() << "success while saving ofImage (using freeimage) " << filename;
    floatimage.resize(floatimage.getWidth() / thumbnailProxy, floatimage.getHeight() / thumbnailProxy);
    filename = prefix + "_proxy." + extension;
    if (!floatimage.save(ofToDataPath(filename, true)))
        ofLogWarning() << "failed while saving ofImage (using freeimage) " << filename;
    else
        ofLogNotice() << "success while saving ofImage (using freeimage) " << filename;
}

template <typename PixelType>
void resizeIOTest_ofPixels(int thumbnailProxy, string prefix, string extension, ofInterpolationMethod interp) {
    ofLogNotice() << "ofPixels test using tinyexr & stb to load and save float images and proxies";
    int width, height, channels;
    auto filename = prefix + "." + extension;
    PixelType* data = ImageIO::loadImage(ofToDataPath(filename, true), width, height, channels);
    ofLogNotice() << "ImageIO::loadImage width: " << width << " height: "<< height << " channels: " << channels;
    ofPixels_<PixelType> pixels;
    pixels.setFromPixels(data, width, height, OF_IMAGE_COLOR_ALPHA);
    filename = prefix + "_copy." + extension;
    ImageIO::saveImage(ofToDataPath(filename, true), width, height, channels, pixels.getData());
    pixels.resize(pixels.getWidth() / thumbnailProxy, pixels.getHeight() / thumbnailProxy, interp);
    width = pixels.getWidth();
    height = pixels.getHeight();
    filename = prefix + "_proxy." + extension;
    ofLogNotice() << "saving resized proxy with width: " << width << " height: " << height << " channels: " << channels << " extension: " << extension;
    ImageIO::saveImage(ofToDataPath(filename, true), width, height, channels, pixels.getData());
}

int main() {
    int thumbnailProxy = 10;
    ofLogNotice();
    ofLogNotice() << "--- one";
    resizeTest<unsigned char>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_NEAREST_NEIGHBOR, ofColor(255, 255, 0, 255));
    ofLogNotice();
    ofLogNotice() << "--- two";
    resizeTest<unsigned char>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_BICUBIC, ofColor(255, 255, 0, 255));
    ofLogNotice();
    ofLogNotice() << "--- three";
    resizeTest<float>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_NEAREST_NEIGHBOR, ofFloatColor(1, 1, 0, 1));
    ofLogNotice();
    ofLogNotice() << "--- four";
    resizeTest<float>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_BICUBIC, ofFloatColor(1, 1, 0, 1));

    ofLogNotice();
    ofLogNotice() << "convert pixels int to float with ofPixels =";
    ofPixels_<unsigned char> pixels;
    pixels.allocate(1920, 1080, OF_IMAGE_COLOR_ALPHA);
    pixels.setColor(ofColor(255, 255, 0, 255));
    {
        ofColor_<unsigned char> color = pixels.getColor(pixels.getWidth() * .5, pixels.getHeight() * .5);
        ofLogNotice() << color;
    }
    ofPixels_<float> floatpixels;
    floatpixels = pixels;
    auto color = floatpixels.getColor(pixels.getWidth() * .5, pixels.getHeight() * .5);
    ofLogNotice() << color;

    ofLogNotice();
    ofLogNotice() << "--- 32 bits linear tif resizeIOTest_ofImage";
    resizeIOTest_ofImage<float>(thumbnailProxy, "Digital_LAD_HD720", "tif");
    ofLogNotice();
    ofLogNotice() << "--- 32 bits linear exr resizeIOTest_ofPixels OF_INTERPOLATE_NEAREST_NEIGHBOR";
    resizeIOTest_ofPixels<float>(thumbnailProxy, "Digital_LAD_HD720", "exr", ofInterpolationMethod::OF_INTERPOLATE_NEAREST_NEIGHBOR);
    ofLogNotice();
    ofLogNotice() << "--- 32 bits linear exr resizeIOTest_ofPixels OF_INTERPOLATE_BICUBIC";
    //resizeIOTest_ofPixels<float>(thumbnailProxy, "Digital_LAD_HD720", "exr", ofInterpolationMethod::OF_INTERPOLATE_BICUBIC);

}

maybe we should use channels to walk though channels channelsFromPixelFormat(pixelFormat);


    case OF_INTERPOLATE_NEAREST_NEIGHBOR:{
        size_t dstIndex = 0;
        float srcxFactor = (float)srcWidth/dstWidth;
        float srcyFactor = (float)srcHeight/dstHeight;
        float srcy = 0.5;
        for (size_t dsty=0; dsty<dstHeight; dsty++){
            float srcx = 0.5;
            size_t srcIndex = static_cast<size_t>(srcy) * srcWidth;
            for (size_t dstx=0; dstx<dstWidth; dstx++){
                //size_t pixelIndex = static_cast<size_t>(srcIndex + srcx) * bytesPerPixel;
                //for (size_t k=0; k<bytesPerPixel; k++){
                size_t pixelIndex = static_cast<size_t>(srcIndex + srcx) * channelsFromPixelFormat(pixelFormat);
                for (size_t k=0; k< channelsFromPixelFormat(pixelFormat); k++){
                    dstPixels[dstIndex] = pixels[pixelIndex];
                    dstIndex++;
                    pixelIndex++;
                }
                srcx+=srcxFactor;
            }
            srcy+=srcyFactor;
        }
    }break;


That looks correct to me (see ofpixel_resize_exrPatch.png)

<img width="455" alt="ofpixel_resize_exrPatch" src="https://github.com/openframeworks/openFrameworks/assets/2362156/030cc673-a57f-4e5c-bee8-49dd869ac06e">

I didn't check the OF_INTERPOLATE_BICUBIC...

Thank you !

janimatic commented 4 months ago

This is the current exr output from bicubic resize :
This is the exr output from bicubic resize when i replace bytesPerPixel by channelsFromPixelFormat(pixelFormat) :

That fixed the pixels indices, but bicubic must have something wrong with accumulated colors in float...

janimatic commented 4 months ago

@dimitre This the result of exr bicubic resize, if i remove the clamping code from ofPixels_::bicubicInterpolate:

    //return std::min(static_cast<size_t>(255), std::max(static_cast<size_t>(out), static_cast<size_t>(0)));
    return out;

janimatic commented 4 months ago

To summarize, here is a diff of my changes made in https://github.com/openframeworks/openFrameworks/pull/7936

diff --git a/libs/openFrameworks/graphics/ofPixels.cpp b/libs/openFrameworks/graphics/ofPixels.cpp
index 44a3f926e..d23f4eeaa 100644
--- a/libs/openFrameworks/graphics/ofPixels.cpp
+++ b/libs/openFrameworks/graphics/ofPixels.cpp
@@ -1328,7 +1328,8 @@ float ofPixels_<PixelType>::bicubicInterpolate (const float *patch, float x,floa
    a20 * x2 + a21 * x2 * y + a22 * x2 * y2 + a23 * x2 * y3 +
    a30 * x3 + a31 * x3 * y + a32 * x3 * y2 + a33 * x3 * y3;

-   return std::min(static_cast<size_t>(255), std::max(static_cast<size_t>(out), static_cast<size_t>(0)));
+   //return std::min(static_cast<size_t>(255), std::max(static_cast<size_t>(out), static_cast<size_t>(0)));
+   return out;
 }

 //----------------------------------------------------------------------
@@ -1361,8 +1362,10 @@ bool ofPixels_<PixelType>::resizeTo(ofPixels_<PixelType>& dst, ofInterpolationMe
                float srcx = 0.5;
                size_t srcIndex = static_cast<size_t>(srcy) * srcWidth;
                for (size_t dstx=0; dstx<dstWidth; dstx++){
-                   size_t pixelIndex = static_cast<size_t>(srcIndex + srcx) * bytesPerPixel;
-                   for (size_t k=0; k<bytesPerPixel; k++){
+                   //size_t pixelIndex = static_cast<size_t>(srcIndex + srcx) * bytesPerPixel;
+                   //for (size_t k=0; k<bytesPerPixel; k++){
+                   size_t pixelIndex = static_cast<size_t>(srcIndex + srcx) * channelsFromPixelFormat(pixelFormat);
+                   for (size_t k=0; k< channelsFromPixelFormat(pixelFormat); k++){
                        dstPixels[dstIndex] = pixels[pixelIndex];
                        dstIndex++;
                        pixelIndex++;
@@ -1391,19 +1394,19 @@ bool ofPixels_<PixelType>::resizeTo(ofPixels_<PixelType>& dst, ofInterpolationMe
            size_t patchIndex;
            float patch[16];

-           size_t srcRowBytes = srcWidth*bytesPerPixel;
+           size_t srcRowBytes = srcWidth*channelsFromPixelFormat(pixelFormat);
            size_t loIndex = (srcRowBytes)+1;
-           size_t hiIndex = (srcWidth*srcHeight*bytesPerPixel)-(srcRowBytes)-1;
+           size_t hiIndex = (srcWidth*srcHeight*channelsFromPixelFormat(pixelFormat))-(srcRowBytes)-1;

            for (size_t dsty=0; dsty<dstHeight; dsty++){
                for (size_t dstx=0; dstx<dstWidth; dstx++){

-                   size_t   dstIndex0 = (dsty*dstWidth + dstx) * bytesPerPixel;
+                   size_t   dstIndex0 = (dsty*dstWidth + dstx) * channelsFromPixelFormat(pixelFormat);
                    float srcxf = srcWidth  * (float)dstx/(float)dstWidth;
                    float srcyf = srcHeight * (float)dsty/(float)dstHeight;
                    size_t   srcx = static_cast<size_t>(std::min(srcWidth-1, static_cast<size_t>(srcxf)));
                    size_t   srcy = static_cast<size_t>(std::min(srcHeight-1, static_cast<size_t>(srcyf)));
-                   size_t   srcIndex0 = (srcy*srcWidth + srcx) * bytesPerPixel;
+                   size_t   srcIndex0 = (srcy*srcWidth + srcx) * channelsFromPixelFormat(pixelFormat);

                    px1 = srcxf - srcx;
                    py1 = srcyf - srcy;
@@ -1412,14 +1415,14 @@ bool ofPixels_<PixelType>::resizeTo(ofPixels_<PixelType>& dst, ofInterpolationMe
                    py2 = py1 * py1;
                    py3 = py2 * py1;

-                   for (size_t k=0; k<bytesPerPixel; k++){
+                   for (size_t k=0; k<channelsFromPixelFormat(pixelFormat); k++){
                        size_t   dstIndex = dstIndex0+k;
                        size_t   srcIndex = srcIndex0+k;

                        for (size_t dy=0; dy<4; dy++) {
                            patchRow = srcIndex + ((dy-1)*srcRowBytes);
                            for (size_t dx=0; dx<4; dx++) {
-                               patchIndex = patchRow + (dx-1)*bytesPerPixel;
+                               patchIndex = patchRow + (dx-1)*channelsFromPixelFormat(pixelFormat);
                                if ((patchIndex >= loIndex) && (patchIndex < hiIndex)) {
                                    srcColor = pixels[patchIndex];
                                }

janimatic commented 4 months ago

PS : sorry for the noise, the latest change for bicubic is not good (it just looks like nearest neighbor with some offset) Just removing the clamping didn't fix it...

dimitre commented 4 months ago

did you paste your latest code here? I've run and I think there is a problem with the alpha channel on the exr copy

janimatic commented 4 months ago

in the the zip file , the main.cpp should be replaced with the code from https://github.com/openframeworks/openFrameworks/issues/6226#issuecomment-2099906869 Other than that i've just adapted the openframeworks source like i said. What are you using to display the exr ? (i use natron)

dimitre commented 4 months ago

opening the copy here in macOS finder and pixelmator

janimatic commented 4 months ago

@dimitre indeed the sample image had a luminance like alpha channel... here is the source exr with white alpha sorry! [Uploading Digital_LAD_HD720.zip…]()

janimatic commented 4 months ago

oops, you're right the copy doesn't copy the correct channel for alpha... alpha=blue...

dimitre commented 4 months ago

new image link appear as uploading here if you agree this pr corrects the main issue (artifacts & crash with resize) I'll be merging it https://github.com/openframeworks/openFrameworks/pull/7936

janimatic commented 4 months ago

well i am not sure it works at all in float if not using channels instead of bytes... But we can always complete the fixes later if you prefer By the way my exr code (ImageIO) adds a problem (blue channel on alpha output...) :

bool saveImageEXR_rgba(const char* outfilename, int width, int height, const float* rgba) {
    std::cout << "saveImageEXR_rgba file " << outfilename << std::endl;
    EXRHeader header;
    InitEXRHeader(&header);
    EXRImage image;
    InitEXRImage(&image);
    image.num_channels = 4;
    std::vector<float> images[4];
    images[0].resize(width * height);
    images[1].resize(width * height);
    images[2].resize(width * height);
    images[3].resize(width * height);

    // Split RGBARGBARGBA... into R, G, B, A layers
    for (int i = 0; i < width * height; i++) {
        images[0][i] = rgba[image.num_channels * i + 0];
        images[1][i] = rgba[image.num_channels * i + 1];
        images[2][i] = rgba[image.num_channels * i + 2];
        images[3][i] = rgba[image.num_channels * i + 3]; // 2]; NO! 
    }

dimitre commented 4 months ago

janimatic commented 4 months ago

Digital_LAD_HD720.zip

dimitre commented 4 months ago

Yes I was suggesting mergin only the pixelsSize issue for now, so I'll be able to test on master later.

dimitre commented 4 months ago

now copy looks OK 🖼️

janimatic commented 4 months ago

yes that's fine for me if you prefer to merge only the pixelsSize for now, even if we know that it creates a channel offset in resize. For the additional changes suggested here, only the bicubic resize of float images remains to be fixed i think. Thanks!

janimatic commented 4 months ago

@dimitre Hello, if you want to keep the unsigned char pixels code intact, you could keep the clamp for all template types for the return of float ofPixels_::bicubicInterpolate like this :

    if (std::is_floating_point<PixelType>::value)
        return std::min(1.0f, std::max(out, 0.0f));
    return std::min(static_cast<size_t>(255), std::max(static_cast<size_t>(out), static_cast<size_t>(0)));

That ensure that there won't be any regression. I suggest applying my patch , plus that change, in you pr, before merging... And then we can try to see if bicubic can be enhanced in float images. Thanks!

dimitre commented 4 months ago

@janimatic yes, PR updated. just to be sure, is it possible to use PixelType double?

dimitre commented 4 months ago

@janimatic I suggest two separate PRs. we can keep this one for just sizeof and clamping, merging and you can submit another with resize corrections

dimitre commented 4 months ago

Hello @janimatic PR was just merged. Feel free to open another PR with resize fixes

https://github.com/openframeworks/openFrameworks/pull/7936

dimitre commented 3 months ago

@janimatic can you please make a PR with your patch, so we can test it ?

janimatic commented 3 months ago

Hello @dimitre , I just did https://github.com/openframeworks/openFrameworks/pull/7989 Eventually some variables names might still be confusing (color indices vs pixels indices) ? Thanks

johanjohan commented 3 months ago

hello guys, thank you for fixing this after such a long time.

dimitre commented 3 months ago

Now merged. can you please test with the latest master to see if this issue can be closed? Thank you all

dimitre commented 3 months ago

@janimatic I've added your code for testing exr, tiff, png images with some changes to this repo https://github.com/dimitre/ofTests and invited you as a collaborator

it would be great if we could maybe support some different bit depth in ofImage. I added two more files, exported from the .exr file from Pixelmator, a 16bit png and one HDR png (I didn't know it existed)

EDIT: feel free to push changes there

danoli3 commented 3 months ago

This is wicked!!! Yeah let's get CI on this ofTests. I has forked

OpenEXR issues looking patched on FreeImage SVN upstream, I'll port those into the Patched Repo. It was just not compatible with the super old LucasArts code, which is updated on Github, just subdependancy issue with FreeImage. FreeImage had to have some subdepends disabled due to incompatibilities to C++14/17 and compile targets.

janimatic commented 3 months ago

@dimitre excellent thanks! Some ideas of things we could add tests for :

BMP, TGA, JPEG, HDR stb image write
JPG, TGA, BMP, PSD (comp), GIF, HDR, PIC stb image read
exr with different compressions / tiles etc...
The bicubic resize didn't look as smooth as the freeimage one if i remember...
In the end it would be great integrate those high dynamic range image i/o to openframeworks base ? complementing FreeImage with TinyExr and StbImage...
short pixels. I never use it though...
I have been using custom/arbitrary structures pixels for some algorithms, but no i/o since it's not standard... (example: mlaa)
For info Stb has a (scary 10000 lines, but maybe usefull for reference) resize single header lib... https://github.com/nothings/stb
Do you consider using cmake ? That would probably make cross-platform testing easier. Cheers! PS there are many other formats used in the industry like dpx etc... but that's a lot of work that has been done already... https://github.com/AcademySoftwareFoundation/OpenImageIO

dimitre commented 3 months ago

Great! it will be amazing if OF could handle more image formats and bit depth. I would love to have HDR io in OF also, and +1 for adding other libs to handle HDR files (if licensing is OK)

Even simple tests like opening your tiff file is failing, so maybe there is more work needed on OF core

dimitre commented 3 months ago

I think the place to have a look is putBmpIntoPixels function on OF Core.

NickHardeman commented 3 months ago

@dimitre, HDR loading should be supported, not sure about saving though. See https://github.com/openframeworks/openFrameworks/tree/master/examples/gl/materialPBRExample

dimitre commented 3 months ago

yess. I suppose it can be done now allocating the right kind of ofPixels and using ofLoadImage right? if we use ofImage_<float> instead it fails when allocating texture

EDIT: My bad, there was no GLFW window on tests, so no OpenGL context for textures

openframeworks / openFrameworks

ofPixels - artifacts / crashes with .resize() #6226

ifdef USE_VEC

else

endif // USE_VEC

ifdef USE_VEC

else

endif // USE_VEC

ifdef USE_VEC

else

endif // USE_VEC

ifdef USE_VEC

else

endif // USE_VEC

ifdef USE_VEC

else

endif // USE_VEC

ifdef USE_VEC

else

endif // USE_VEC

ifdef USE_VEC

else

endif // USE_VEC