Get image pixels from Framebuffer

Hi Brad, I am sry. I think this not an issue, but I hope that you could help me with it.

I want to connect dlib in GPUImage2, draw facemasks (similar to MSQRD app) in camera preview and capture video. I found existed samples, containing DlibWrapper for iOS. That wrapper helps to generate 2D array from image buffer:

dlib::array2d<dlib::bgr_pixel> img;

CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
CVPixelBufferLockBaseAddress(imageBuffer, kCVPixelBufferLock_ReadOnly);

size_t width = CVPixelBufferGetWidth(imageBuffer);
size_t height = CVPixelBufferGetHeight(imageBuffer);
char *baseBuffer = (char *)CVPixelBufferGetBaseAddress(imageBuffer);

img.set_size(height, width);
img.reset();
long position = 0;
while (img.move_next()) {
    dlib::bgr_pixel& pixel = img.element();
    long bufferLocation = position * 4; //(row * width + column) * 4;
    char b = baseBuffer[bufferLocation];
    char g = baseBuffer[bufferLocation + 1];
    char r = baseBuffer[bufferLocation + 2];
    dlib::bgr_pixel newpixel(b, g, r);
    pixel = newpixel;
    position++;
}

and deep detect face shape points inside previously detected face rects:

dlib::rectangle oneFaceRect = detectedFaceRectItem
dlib::full_object_detection shape = dlib::shape_predictor(img, oneFaceRect);

draw detected point on generated 2D pixel array and put it back to buffer;

Looks simple, but is it possible to use it inside didOutputSampleBuffer Camera callback: https://github.com/BradLarson/GPUImage2/blob/master/framework/Source/iOS/Camera.swift#L169

Previously I thought that I have to modify cameraFrame object inside this method. But I believe it is impossible because sampleBuffer captured with using kCVPixelFormatType_420YpCbCr8BiPlanarFullRange pixel buffer format and I don't want to change it to kCVPixelFormatType_32BGRA.

Next I found that GPUImage code contains convertYUVToRGB(): https://github.com/BradLarson/GPUImage2/blob/master/framework/Source/iOS/Camera.swift#L233

Maybe I have to modify resultBuffer? I believe this is the final object before rendering and it contains everything what I need for generating 2D pixels array for dlib, drawing detected face shape points and modify this resultBuffer before rendering?

Can you help me with 2D pixels array generation and modifying resultBuffer before rendering?

Thank you.

BradLarson / GPUImage2

Get image pixels from Framebuffer #169