MTLWriteTextureToFile() no longer works

[neurolabusc]

The function MTLWriteTextureToFile() no longer works. Tested on my MacBook 10.14.6. I would appreciate any thoughts or solutions you might have.

For example, run this project and choose File/Save. A bitmap is created, with a size that corresponds to the form size. However, the bitmap is blank. It sounds like this problem.

I think this is also related to this comment (with Pascal code) that Without calling synchronizeResource() offscreen rendering will NOT work on Macs with dedicated graphics (AMD)..

[neurolabusc]

Ryan The code from Igor Kokarev helped me resolve this. Changing the following procedures will resolve this issue. I am happy to make a pull request if you want. I think the resulting code is cleaner, as it removes the mysterious number of refreshes to view.draw. As an added bonus, passing an empty string as the file path will have the function save the image to the clipboard, rather than saving it to disk. I would be grateful if you could review this code and insert it into your great package.

procedure MTLWriteTextureToFile(path: pchar; fileType: NSBitmapImageFileType = NSPNGFileType; imageProps: NSDictionary = nil);
begin
  Fatal(CurrentThreadContext = nil, kError_InvalidContext);
  with CurrentThreadContext do begin
    view.setFramebufferOnly(false);
    //view.draw;
    //view.draw;
    //view.draw;
    MTLWriteTextureToFile(view.currentDrawable.texture, path);
    view.setFramebufferOnly(true);
  end;
end;

procedure MTLWriteTextureToFile(texture: MTLTextureProtocol; path: pchar; fileType: NSBitmapImageFileType = NSPNGFileType; imageProps: NSDictionary = nil);
var
  width, height, bytesPerRow, bytesCount: integer;
  bytes: pointer;
  colorSpace: CGColorSpaceRef;
  bitmapInfo: CGBitmapInfo;
  provider: CGDataProviderRef;
  imageRef: CGImageRef;
  finalImage: NSImage;
  imageData: NSData;
  imageRep: NSBitmapImageRep;
  blitEncoder: MTLBlitCommandEncoderProtocol;
  pb : NSPasteboard ;
begin
    Fatal(texture.pixelFormat <> MTLPixelFormatBGRA8Unorm, 'texture must be MTLPixelFormatBGRA8Unorm pixel format.');
    // read bytes
    width := texture.width;
    height := texture.height;
        bytesPerRow := texture.width * 4;
    bytesCount := width * height * 4;
    bytes := GetMem(bytesCount);
        CurrentThreadContext.commandBuffer := CurrentThreadContext.commandQueue.commandBuffer;
        blitEncoder := CurrentThreadContext.commandBuffer.blitCommandEncoder;
        CurrentThreadContext.view.draw;
        blitEncoder.synchronizeResource(texture);
        blitEncoder.endEncoding;
        CurrentThreadContext.commandBuffer.WaitUntilCompleted;
        texture.getBytes_bytesPerRow_fromRegion_mipmapLevel(bytes, bytesPerRow, MTLRegionMake2D(0, 0, width, height), 0);
        colorSpace := CGColorSpaceCreateDeviceRGB;
    bitmapInfo := kCGImageAlphaFirst or kCGBitmapByteOrder32Little;
    provider := CGDataProviderCreateWithData(nil, bytes, bytesCount, nil);
    imageRef := CGImageCreate(width, height, 8, 32, bytesPerRow, colorSpace, bitmapInfo, provider, nil, 1, kCGRenderingIntentDefault);
    finalImage := NSImage.alloc.initWithCGImage_size(imageRef, NSMakeSize(width, height));
    imageData := finalImage.TIFFRepresentation;
    imageRep := NSBitmapImageRep.imageRepWithData(imageData);
    //imageProps := NSDictionary.dictionaryWithObject_forKey(NSNumber.numberWithFloat(1), NSImageCompressionFactor);
    imageData := imageRep.representationUsingType_properties(fileType, imageProps);
        if strlen(path) < 1 then begin
           pb := NSPasteboard.pasteboardWithUniqueName();
           pb :=  NSPasteboard.generalPasteboard;
           pb.clearContents;
           pb.writeObjects(NSArray.arrayWithObjects_count(@finalImage, 1)) ;
        end else
        imageData.writeToFile_atomically(NSSTR(path), false);
    CFRelease(provider);
    CFRelease(imageRef);
    CFRelease(colorSpace);
    finalImage.release;
end; 

procedure MTLEndCommand (waitUntilCompleted: boolean = false);
begin
    with CurrentThreadContext do begin
    if kMetalContextFrameStateRender in frameState then
        commandBuffer.presentDrawable(CurrentThreadContext.view.currentDrawable);
    commandBuffer.commit;
    if waitUntilCompleted then
        commandBuffer.waitUntilCompleted;
    //commandBuffer := nil; //20200622
    frameState := [];
    end;
end; 

[genericptr]

Finally took the time to look at this. I'm seeing I had some junk comments left in MTLWriteTextureToFile which suggested something wasn't correct they way I did it but settled anyways because it worked (then). It sounds like this is the correct solution but I need to get these demos running on Catalina first and I'm seeing various things broke since I built it last. Also I'm getting crashes on this code right now so I need to confirm it works at all on Catalina but I can confirm taking the screenshots stopped working.

Copying to the pasteboard should be another function instead of mixing it in to MTLWriteTextureToFile but I'll stub that out when I get this working.

[neurolabusc]

Great! Given the MacOS move to Arm CPUs, one worries that the deprecated OpenGL will not be present in these new computers (as Apple's iOS/iPad only supported OpenGL ES). Your framework seems timely.

1.) I know you like to code in straight Pascal, but my Lazarus demos show my screen capture is working. I still run on Mojave, so I would be grateful if you could test these. 2.) For you straight Pascal code, you might also want to consider porting Igor's offscreen demo to your Framework. It does show the compute potential for Metal. 3.) The function MTLWriteTextureToFile(path: pchar; fileType: NSBitmapImageFileType = NSPNGFileType; imageProps: NSDictionary = nil) never uses the fileType and imageProps variables. These input variables should either be removed or passed on to the overloaded call of MTLWriteTextureToFile.

[genericptr]

Yes, I got it working now so I'll make a commit with changes. The OBJ reader examples broke on Catalina but I may just leave it like that for now. I think the best approach is to make a new function which returns the raw image data and then some helper functions to write to file or pasteboard so the user has full control.

[genericptr]

I think this is fixed in the latest commit now. I only tested on 10.15 and FPC trunk 3.3.1 so test and see if it works on your system.

[neurolabusc]

Ryan This works great on 10.14. For clarity, can I suggest you remove the unused variable fileType: NSBitmapImageFileType; imageProps: NSDictionary from the Clipboard functions. Likewise, you can remove the local variables imageData: NSData; imageRep: NSBitmapImageRep; The resulting function would look like this:

procedure MTLWriteTextureToClipboard(texture: MTLTextureProtocol);
var
  imageRef: CGImageRef;
  finalImage: NSImage;
  pb: NSPasteboard;
begin
    imageRef := MTLCopyLastFrameTexture(texture);
    if imageRef <> nil then
        begin
            finalImage := NSImage.alloc.initWithCGImage_size(imageRef, NSMakeSize(CGImageGetWidth(imageRef), CGImageGetHeight(imageRef)));

            pb := NSPasteboard.generalPasteboard;
            pb.clearContents;
            pb.writeObjects(NSArray.arrayWithObject(finalImage));

            finalImage.release;
            CFRelease(imageRef);
        end;
end;

diff.txt

[genericptr]

I left those in as default parameters so you can ignore them if you want. No good? I thought the user may want to have control over what the actual format is instead of always getting a PNG.

[neurolabusc]

@genericptr for the Clipboard function they do nothing. The clipboard is provided with the NSImage, not the NSData. Therefore, the fileType and imageProps settings have no impact on what is saved to the clipboard. The clipboard gets the original data representation. So if the user wants to set them, they will be disappointed to see they have no influence.