[RFC] BAL: Graphics API framework

[msiglreith]

Introduction

We have multiple (upcoming) clients of low level gfx-rs with strong performance or validation constraints. Our current HAL interface strongly adapts the Vulkan API together with the constraints burdened upon the backends (e.g pipeline layout compatibility).

gpuweb will have a higher level API giving room for backend specific optimization without having to go through the Vulkan/HAL bottleneck. Additionally, it will require specific validation steps in contrast to other APIs. Other clients like Amethyst or WebRender could also benefit from optimization in case of higher level or small specialized interfaces.

Concept (BAL)

Gradually lift the bottleneck imposed by HAL by adding a smaller domain specific framework for building graphics API on top of the lower level APIs. This framework, called BAL, should be seen as floating target not stabilizing anytime soon (or at all) and provide basic backend-specific and agnostic primitives for constructing higher level APIs (portability, WR, gpuweb). These primitives might be special cache structures required for only a part of the frontend APIs.

Implementation

The transition towards the new BAL structure can happen gradually by starting to publically expose more primitives from the backends, shifting HAL slowly towards a higher level frontend API for Rust users only and allow users to hookup a basic Vulkan library (e.g Ash) to be able to start working with a more or less fixed API.

Graphics API implementation

Being able to quickly implement and optimize a graphics frontend API (e.g gpuweb) should be the main goal of BAL. A library author could start working with the Rust Vulkan library for implementation of the Vulkan path, automatically gaining support for the other backends due to portability initiative. Certain fast paths can be sped up writing backend specific paths on top of the primitives provided by BAL.

Future of HAL

HAL would be mainly interesting as safer and rustic wrapper of Vulkan or moved more towards a middleground of all APIs. It should be seen as low overhead frontend but not as the only API exposed from gfx-rs. Future higher levels abstractions like gpuweb could be more appealing for certain user groups due to slimmer and simpler interface.

[zakarumych]

So BAL will be a collection of APIs each backend supports only partially. And HAL will be essential subset of BAL that is supported by all backends. Am I understand this correctly?

[msiglreith]

@omni-viral At the beginning, yes. I'm feeling that we should shift HAL towards the 'frontend' side and treat it like WebRender, gpuweb and portability which basically just stack upon the primitives provided by BAL as I would like dropping the Vulkan backend completely as BAL backend and users (and ourselfes) just directly act upon Vulkan. So essentially BAL would be some kind of utility library over D3D12, Metal, GL, ... thin wrapping components with identical semantics and also directly exposing certain features (e.g possibly RootSignatures or DescriptorsHeaps on d3d12 side) by the backend crates.

// portabilty draw command
fn gfxCmdDraw(..) {
   #[cfg(features = "dx12")]
   {
       buffer.user_data.flush();
       buffer.command_list.draw();
   }
   #[cfg(features = "vulkan")]
   {
       buffer.draw();
   }
   ...
}

// gpuweb draw command
fn gwCmdDraw(..) {
   validate();
   #[cfg(features = "dx12")]
   {
       // insert transition, UAV, aliasing barriers possibly
       buffer.validate_dx12();
       buffer.command_list.draw();
   }
   #[cfg(features = "vulkan")]
   {
       // insert pipeline barriers
       buffer.validate_vk();
       buffer.draw();
   }
   ...
}

The latest discussion about RawC, Ash, HAL, .. was often influenced by what our main user base will maybe look like and trying to somehow satisfy all their constraint but IMO that's not really possible. Therefore shifting about more of the complexity onto the clients (similar to the low level APIs heh). At the end users will choose which API/frontend fits best for them.

[zakarumych]

@msiglreith Could you also add a snipped for user who wants to write code once for all platforms.

[zakarumych]

Would it be possible for user with backend-agnostic code to add some backend-specific code without rewriting stuff all over the place?

[msiglreith]

@omni-viral For writing backend agnostic code on top one of the 'frontends' (HAL, Ash, gpuweb in the future) should be used. 'BAL' is more like an implementation detail/layer allowing us to provide better support for other APIs. Somewhere between a Rust D3D12/Metal/.. wrapper and current HAL.

The easiest way for being able to add backend specific paths would be via Vulkan(Portability)/Ash due to it's special status of being backend and frontend at the same time. So in the first iteration relaying on the portability implementations and in further iterations add backend specific parts. HAL for example could also expose some backend specifics but this would then go through the HAL API as some kind of extension.

The difficulty with backend specific paths when building on top of frontends whould be requiring us to expose internals of the frontend implementation. I don't think BAL would have some sort of 'Backend' trait with different resource traits but rather frontends building for example the CommandBuffer via some primitives:

// HAL: dx12
pub struct CommandBuffer {
   command_list: bal::dx12::CommandList,
   pipeline_cache: bal::dx12::PipelineCache,
   gr_user_data: bal::dx12::UserDataCache,
   cp_user_data: bal::dx12::UserDataCache,
   ...
}

impl hal::CommandBuffer for CommandBuffer {
  fn draw() {
     self.gr_user_data.flush();
     self.command_list.draw();
  }
 ...
}

// gpuweb dx12:
pub struct CommandBuffer {
   command_list: bal::dx12::CommandList, // Or a deferred command list ..?
   pipeline_cache: bal::dx12::PipelineCache, // for example gpuweb using 2 slot model but no pipeline layout compatibility
   gr_state_context: bal::dx12::StateValidator, // do some validation stuff on graphic action commands
   ... 
}

impl gpuweb::CommandBuffer for CommandBuffer {
   fn draw() {
     self.transition_resources(Cmd::DRAW);
     self.gr_state_context.validate();
     self.command_list.draw();
  }
   }
   ...
}

[zakarumych]

So if I use hal there wouldn't be easy way to write backend specific code or hal will expose underlying structures in non-generic environment. Like if I have instance of gfx_backend_dx12::CommandBuffer instead of opaque <B as Backend>::CommandBuffer will I be able to access dx12 specific methods?

[Elzair]

If I wanted to write cross-platform graphics code, I should use the Vulkan portability library. Internally, portability will map its commands to either the Ash Vulkan wrapper or BAL (which can use either Direct3D12 or Metal {or OpenGL 2}). Is this correct?

[msiglreith]

So if I use hal there wouldn't be easy way to write backend specific code or hal will expose underlying structures in non-generic environment.

That's rather a design decision on how HAL would look like at the end (just a rustic vulkan wrapper?). But I would rather avoid exposing internals and I don't see that much gain from it. We can also think later on how we want to define some extensions to provide better support for backend details.

If I wanted to write cross-platform graphics code, I should use the Vulkan portability library.

Any of the mentioned frontend libraries:

• HAL: API not fixed but more rustic
• Vulkan PI: Fixed API but more like raw C bindings
• (WebRenderer: Very domain specific frontend)
• gpuweb-rs: In the future, higher level than Vulkan but probably easier to use

Internally, portability will map its commands to either the Ash Vulkan wrapper or BAL

Exactly! To clarify, I don't think BAL will look like the current HAL in terms of having some shared trait implemented by all backends. Rather portability will map to Vulkan, BAL-D3D12, BAL-Metal, BAL-OpenGL. The BAL prefix means some zero abstraction wrappers above the native backend APIs + common primitives for easing the implementation of the frontend APIs.

[anderejd]

I'm a little confused by this thread, here are some questions.

Will HAL be supported or will it go away? Is HAL considered a failure or is it still the future of graphics programming in rust?

[kvark]

@anderejd this is a non-trivial topic, it's totally ok to be confused. Let me clarify

Will HAL be supported or will it go away?

HAL will be supported, see this item in the frontend list of the previous comment:

HAL: API not fixed but more rustic

Is HAL considered a failure or is it still the future of graphics programming in rust?

We've been giving HAL an evaluation in #2206, and the conclusion (wasn't obvious at all!) we reached is to move on with HAL that is built using BAL components.

It's hard to call it "the future of graphics programming in rust" just yet. There are many different clients and requirements. Less demanding users would just use existing engines. A bit more demanding would use something like WebGPU native library. Most demanding would use HAL. And super hardcore would use BAL. As you can see, it's not clear how much space there is for HAL in this scheme. At least we can run Warden tests on it and provide examples, plus Amethyst and WebRender already have ports on it, so it is useful.

[anderejd]

That clears up my confusion, thank you! :)

[kvark]

@msiglreith I liked the idea from the start, at least on paper, but the more I think about it, the more I'm getting confused. The main question is - what are the specific BAL blocks we'd be able to expose? Without understanding this, it's hard to see the whole picture working out.

Presumably, BAL would have blocks shared between 2 or more APIs, or otherwise these blocks would just be in the backend code only (public). One such area I can think of is resource binding. D3D11 has slot-based resource binding, and Metal does. But they are so different at the same time that it's not clear if it's even worth a try unifying them. The binding slot geometry is different, the interaction with argument buffers adds a lot of spice. It feels like, if one to write this logic block interface in a generic way then we'd just end up with an over-engineered abomination.

And if we don't try to share the logic blocks between backends, then there is no BAL: there would just be some APIs in each backend specifically. I.e. gfx-backend-metal would just have a few helpers on top of metal-rs, gfx-backend-dx12 would have a few helpers on top of d3d12-rs, and so on.

[msiglreith]

Has been some time (:

From a more outside view, wgpu is the main driving force regarding features and also API changes. As I see it right now, there is basically wgpu and vk-portability as consumer, while most users pick wgpu as unified api layer. HAL and WR themself are probably way more niche in this regard.

what are the specific BAL blocks we'd be able to expose

One point was to provide more 'extension' like functions or cargo features to allow alternative paths. For example pipeline barriers are really different between vulkan (usually global barriers) and dx12 (resource state changes). Tying it to on or the other model hurts the other backend in terms of performance. Some of the transfer operations like blitting or unaligned copies may not be needed by wgpu at all and could also be moved to vulkan portability at the end.

2 years later I would advocate on stronger tying hal to wgpu and moving more away from HAL <-> Vulkan interop or general see it as strongly user facing api (which kind of blurs the HAL/BAL approach).