Built-in kernel registry

[pjaaskel]

This is to store any comments/progress/feedback related to the idea of a "built-in kernel registry" which would be an index of semantically well-defined built-in kernels. HW vendors can freely choose a subset of built-in kernels (BiKs) to implement and accelerate as fixed function "black box" functions. The clients can then pick up vendor-advertised BiKs that match their application's acceleration needs. The idea has been presented in OpenCL work-group meetings and received at least some lukewarm interest/curiosity.

The extension specification could be named cl_khr_defined_biks or similar and would simply add a sentence to the clGetDeviceInfo()'s CL_DEVICE_BUILT_IN_KERNELS which declares along the lines of

"The built-in kernels returned by the device of which names start with 'khr.' adhere to the defined semantics and behavior stored in the Khronos Built-in Kernel Registry located in http://.../KhronosGroup/BiK-Registry."

For the concept to make sense as a Khronos-entity and a cross-vendor portability feature, the built-in kernel abstraction should be used more extensively (hundreds of entries in the registry, not just a couple) and their semantics should be a bit more generic than very specific exact matches to the underlying single-vendor hardware. This is to create better chances of matching with the application level task graphs instead of becoming a collection of vendor-specific BiKs.

The "living" slide deck that tries to explain the concept and collects the open questions can be seen here.

All kind of feedback / comments appreciated!

[bashbaug]

One thought: it would be interesting to write this up first as-if it were an extension, similar to cl_intel_motion_estimation. It's still not entirely clear to me what the benefit would be of the built-in kernel registry vs. reusing extension specifications for this type of documentation, and maybe seeing an extension would help to emphasize the differences.

[pjaaskel]

To match the flexibility of the proposal with extensions (the proposal allows choosing built-in-kernels freely from implementor and user perspective), there would need to be an extension for each built-in-kernel (not a set of, but each single kernel) which the vendors can implement and clients call from their app if the device advertises them. It works at the moment when there are only a few BiKs, but part of this idea is to have hundreds of potentially HW accelerated functions for key areas, thus a separate organized index where BiKs could refer to each other and have bit accurate machine readable semantics would help automated graph lowering and other use cases. But like I said, it is not useful if BiKs won't become used extensively and there are only a few of them supported by vendors, so it's a bit of a chicken-egg at this stage.

[bashbaug]

This is still a bit hand-wavy I know, but I don't think we would necessarily need an extension for each built-in kernel, and in the limit we could be OK with a single extension for a living set of built-in kernels. Here's how it could work:

• We have one single extension, name tbd, could be something like cl_khr_defined_biks as described above.
• The extension will define the semantics of a set of built-in kernels from a defined "namespace" - probably some sort of built-in kernel name prefix or suffix.
• The set of built-in kernels could be considered "living", meaning that subsequent versions of the extension could do things like add new built-in kernels (almost certainly) or add additional behavior or capabilities to already-defined built-in kernels (possibly). I would not recommend changing the behavior of already-defined built-in kernels in a subsequent version of the extension because this would break backward compatibility, though we could give ourselves the freedom to do so with a major version increment if needed.
• If a device/platform declares support for this extension is is making two promises:
  • It will not define any built-in kernels from the namespace for its own use. In other words, the namespace will be considered reserved for built-in kernels defined by the extension.
  • When it declares support for a built-in kernel described in the extension it will adhere to the semantics described by that version of the extension specification.
  • Note that we can test both of these promises with a CTS test.

Here's how an application would use this extension:

• It would first check for support for the extension, and optionally the version of the extension if the application requires support for a behavior added in a specific version of the extension. This is the only extension check the application needs to perform.
• After checking for support for the extension, the application would query for support for the built-in kernels it needs. If the built-in kernel name is supported, the application can use it with confidence, otherwise the application could gracefully fail or uses a substitute non-built-in kernel instead.

Would this work?

[pjaaskel]

Thanks for the feedback. Yes, it would basically be fine to put the "BiK registry" to a single extension which includes the flexibility aspect and is expanded when new BiKs are added and old ones updated.

The main motivations for a separate "registry" (basically a structured git repository) would be the semantics definitions and machine readability: If the semantics (if non-trivial to describe bit-accurately in words unlike "matrix multiplication") are written in OpenCL C or SPIR-V, the extension document becomes bloated. Machine readability calls for a structured format such as XML or JSON from which the human-readable documentation could be also generated.

Anyhow, the first version could do without these fancy features and we can add them later along with the possible separate git repo, if the idea receives support and a lot of BiKs get added to the extension.

Another topic to consider: Do we want to allow making the BiKs also callable from software-defined kernels like function calls? It would be useful in many cases where the BiK is fine grained or to simply reduce the number of commands to launch (to support manual or automated operator/kernel fusion).

[jansol]

Another topic to consider: Do we want to allow making the BiKs also callable from software-defined kernels like function calls? It would be useful in many cases where the BiK is fine grained or to simply reduce the number of commands to launch (to support manual or automated operator/kernel fusion).

That's an interesting question also in light of recent features like VK_NV_device_generated_commands / D3D12 ExecuteIndirect on the graphics API side.

[jansol]

Another interesting possibility would be to make BiKs eligible for use with the enqueue_kernel(...) function from OpenCL C 2.0.

[pjaaskel]

@jansol yep, same thought came to mind when I started typing this down as an extension yesterday. The first draft as html here. Please comment here.

[bashbaug]

Another interesting possibility would be to make BiKs eligible for use with the enqueue_kernel(...) function from OpenCL C 2.0.

I had a similar reply half typed out also (great minds think alike?).

Can a built-in kernel be recorded into a command buffer (cl_khr_command_buffers)? This might "just work", but I don't recall discussing this use-case, so it would be good to verify.

[pjaaskel]

@bashbaug yes, being able to benefit from cmdbuffers is one of the great benefits of being able to define all core functionality of the program in CQs. I mentioned about that in the above extension draft. If that's currently not possible for some reason for the command buffer extensions, it shall be changed, as that's been the master plan all the time from our side (in the research group's activities I mean).

[pjaaskel]

The next revision (led by Henry) is being discussed here: https://github.com/pjaaskel/OpenCL-Docs/pull/1

The HTML render of the latest WiP draft.

[pjaaskel]

This issue is now outdated. Let's continue the discussion in https://github.com/KhronosGroup/OpenCL-Docs/pull/867.