Add Metal support (API by Apple)

[ihhub]

We have support of CUDA and OpenCL which are wide-spread technologies. The code for them locates in src/cuda and src/opencl directories respectively. We would like to add support for Metal - API introuduced by Apple Inc. exclusively for iOS and MacOS. For a person who wishes to start working on this issue there is a proposed way to do:

1. Create a new image buffer class with unique id as 4 and store it in src/metal/image_buffer_metal.h file
2. Add few functions (BitwiseAnd for example) into newly created src/metal/image_function_metal.h and src/metal/image_function_metal.cpp files.
3. Create files src/metal/metal_device.h and src/metal/metal_device.cpp to manage Metal devices
4. Create files src/metal/metal_helper.h and src/metal/metal_herlper.cpp to manage Metal devices for kernels

This is quiet big task so please don't try to do in one shot :)

[scrounchtike]

Hi, this is my first post/comment on penguinV. 0x72D0 introduced me to this open source project and I think I will start with this feature request. Just a head's up that Ill start working on it : )

[ihhub]

Hi @scrounchtike thanks for picking this task! We're looking forward for your work.

Just to simplify your work I highly recommend to have a look into OpenCL implementation as it's very close technology. Please don't try to make in one shot, you could do in a small commits as well :)

[yannlabou]

Isn’t the Metal API written in Swift and Objective-C?

[ihhub]

You could combine Objective-C and C++ compilation.

[scrounchtike]

yes Metal is Objective-C, but I will probably end up making a Objective-C++ wrapper for accessing the Metal API functions.

[ihhub]

We don't need to access Metal API directly as we could hide all logic inside source files. From my personal experience it is possible to combine Objective-C and C++ for iOS/MacOS applications.

[scrounchtike]

Sorry Im not sure I understand the first sentence you wrote. "We don't need to access Metal API directly" and "we could hide all logic inside source files" do not make much sense to me. Could you elaborate please?

But in short yes, the wrapper I was referring to would be Objective-C and C++ so that our code could remain C++ outside of said wrapper. Note that Im not suggesting to "hook" the Metal API C functions for direct access. We don't really have a choice to use the Objective-C dispatcher from what Ive read quickly on the web.

Ill try to make a skeleton of the architecture this week for starters : )

[ihhub]

I meant that we need to uae wrappers or something like this. Let's wait for your first implementation and then we would have a discussion if needed :)

[ihhub]

Hi @scrounchtike how is it going? :)

[scrounchtike]

It's going well, thanks for asking.

So far, I implemented grayscale using Metal and a few utilities for texture uploading/downloading.

One thing I am confused about is how the GPU images (textures) are represented in penguinV. Images (PenguinV::Image) are used for representing CPU images as well as GPU textures.

I dont know much about CUDA or OpenCL, but Metal uses its own representation of images (MTLTexture) (its a graphics API after all) so Im not sure I will be able to return a PenguinV::Image containing the image data pointer like the two APIs do! I dont have a pointer to the GPU image data, and even if I would, Metal works on MTLTextures, not raw char* data.

I hope this was somewhat clear. I will probably just change the signatures of image_function.cpp/.h of Metal to use a custom Texture for now until we figure out a better way.

Any ideas to solve this problem are welcome !

[ihhub]

Sorry for late reply. Please take a look into src/opencl/image_buffer_opencl.h and it's class implementation. OpenCL doesn't have raw pointer as well. The solution is to use reinterpret_cast.

[ihhub]

Hi @scrounchtike any news? :)