google / filament

Filament is a real-time physically based rendering engine for Android, iOS, Windows, Linux, macOS, and WebGL2
https://google.github.io/filament/
Apache License 2.0
17.59k stars 1.86k forks source link

Issue with PixelBufferDescriptor on Metal #7656

Closed schirrmacher closed 5 months ago

schirrmacher commented 6 months ago

Desktop (please complete the following information):

I want to render frames of an OpenCV camera stream with PixelBufferDescriptor like this:

void CameraDeviceView::updateFront(cv::Mat frame) {

    if (frame.type() != CV_32FC3) {
        throw std::runtime_error("CV_32FC3 expected");
    }

    auto buffer = Texture::PixelBufferDescriptor(
            frame.data,
            size_t(frame.cols * frame.rows * frame.channels() * sizeof(float)),
            Texture::Format::RGB,
            Texture::Type::FLOAT,
            nullptr
    );
    frameFrontTexture->setImage(*engine, 0, std::move(buffer));
    frameFrontTexture->generateMipmaps(*engine);
}

Is that the right way to do it or do I have to do it with an external image by utilising setExternalImage?

Additionally I applied thread sanitizer and got the following issue:

ThreadSanitizer:DEADLYSIGNAL
==31312==ERROR: ThreadSanitizer: SEGV on unknown address 0x0001599a8000 (pc 0x0001052314b8 bp 0x00016b9b6fc0 sp 0x00016b9b6bd0 T11315128)
==31312==The signal is caused by a READ memory access.
    #0 filament::backend::reshape(filament::backend::PixelBufferDescriptor const&, filament::backend::PixelBufferDescriptor&) <null>:240130048 (neo:arm64+0x1008754b8) (BuildId: a0048f1b079e3dd88dacd53337e570d032000000200000000100000000000b00)
    #1 _pthread_start <null>:232215104 (libsystem_pthread.dylib:arm64e+0x7030) (BuildId: daf953735de639a1a6ced87f3f0629cc32000000200000000100000000010e00)
    #2 thread_start <null>:232215104 (libsystem_pthread.dylib:arm64e+0x1e38) (BuildId: daf953735de639a1a6ced87f3f0629cc32000000200000000100000000010e00)

==31312==Register values:
 x[0] = 0x0000000157a04000   x[1] = 0x0000000109200720   x[2] = 0x0000000000000003   x[3] = 0x0000000000001052  
 x[4] = 0x0000000063000000   x[5] = 0x0000000000000000   x[6] = 0x0000000139e0dfc0   x[7] = 0x0000000000000000  
 x[8] = 0x00000001599a8000   x[9] = 0x0000000000029400  x[10] = 0x000000003f800000  x[11] = 0x0000000159714000  
x[12] = 0x0000000000000000  x[13] = 0x0000000000000000  x[14] = 0x0000000000000002  x[15] = 0x000000000000001f  
x[16] = 0x00000000000000c5  x[17] = 0x0000000109244230  x[18] = 0x0000000000000000  x[19] = 0x000000016b9b6ca0  
x[20] = 0x000000012304b0d8  x[21] = 0x0000000001fa4000  x[22] = 0x0000000000000017  x[23] = 0x0000000000000006  
x[24] = 0x00000000017bb000  x[25] = 0x0000000000000000  x[26] = 0x0000000000000000  x[27] = 0x0000000000000000  
x[28] = 0x0000000000000000     fp = 0x000000016b9b6fc0     lr = 0x0000000105231494     sp = 0x000000016b9b6bd0  
ThreadSanitizer can not provide additional info.
SUMMARY: ThreadSanitizer: SEGV (neo:arm64+0x1008754b8) (BuildId: a0048f1b079e3dd88dacd53337e570d032000000200000000100000000000b00) in filament::backend::reshape(filament::backend::PixelBufferDescriptor const&, filament::backend::PixelBufferDescriptor&)+0x1b0
==31312==ABORTING

Is this issue related to how I use the pixel buffer or is this a bug?

pixelflinger commented 5 months ago

frame is allocated on the stack and you're passing it to PixelBufferDescriptor. It will go out of scope when you exit updateFront.