Web GL

[fitzgen]

Can we make something like glium for Web GL? Does it make sense to jump straight there, or should we build an incremental Rustier version of the Web GL APIs before trying to make an Opinionated(tm) library?

[grovesNL]

IMO it might be useful to have unified OpenGL, OpenGL ES, and WebGL bindings, similar to the unified bindings Emscripten provides. I already started working on this in https://github.com/grovesNL/glow for use in gfx-hal's GL backend (gfx-backend-gl) and I'll continue to expand it as I work through WebGL support for gfx-hal.

With this approach, projects like gfx-hal, glium etc. could use and share these unified bindings instead.

[asny]

After reading the Gloo update, I will suggest to start with a minimalistic WebGL and WebGL2 wrapper which have three main goals:

• Wrap the web-sys functions in nice Rust-y types and interfaces and with a 1-1 correspondence to the WebGL API.
• Elegantly handle as many error cases as possible without any significant overhead.
• Ensure performance wherever it is possible for example by ensuring the state is not changed if the specified state is the same as the current.

.. So basically the same purpose as Glium but without the simplification of the interface. I think it is important to have the thin Rust-y wrapper on top of WebGL as the innermost layer of the onion and when that is done, there is plenty of room to add outer layers. This could be a simplified and safer interface or the combined OpenGL+WebGL API suggested by @grovesNL.

Please don’t hesitate to disagree with some or any of my opinions :-)

[David-OConnor]

What do y'all thinking of mimicking syntax from GFX/Vulkan when able?

I agree that the first step should be a thin wrapper. For later high-level APIs, it would be nice to integrate matrix-manipulation tools. Using cgmath as a dependency is one option, or more generally, ndarray. Or creating our own, with an emphasis on interop with JS TypedArrays.

[yoshuawuyts]

I have no opinions here, other than that regl.party has proven to be a useful model to build longer-lasting WebGL applications in JS. Perhaps it's something to draw inspiration from when we eventually build the higher-level layers.

[asny]

Here’s a more detailed proposal of the basic WebGL crate after request by @fitzgen.

Summary

Safe, rusty wrapper for the web-sys WebGL functionality.

What is included?

• Only standard rust types in the interface.
• Type safe interface.
• Error handling by returning results from all functions that can cause a runtime error.
• Simplification of the interface wherever it does not limit the functionality. (note that this opinion differ a bit from my previous post, I think it is not possible to have a 1-1 correspondence to the WebGL API, but it should be as close as possible)

What is not included?

• WebGL is a state machine which is not really aligned with the rust way of thinking. Unfortunately, there is not a straight forward and unbiased way to handle that. Therefore, I suggest to leave that to the higher level crates.
• Performance is ensured only when it is really easy to do, nothing more.

Motivation

The web-sys WebGL functionality is difficult to approach because of the JS types, conversions and memory management. Also, the WebGL API in itself is very far from the Rust way of thinking, for example is it not type safe at all. So the purpose of this crate is to expose a simple and safe API which is similar, though not identical, to the WebGL API. The intended users are graphics programmers at all levels of experience.

Detailed explanation

Here’s an example implementation:

use web_sys;

// There should be a struct for WebGL 2 context too.
pub struct WebGlRenderingContext
{
    context: web_sys::WebGlRenderingContext
}

impl WebGlRenderingContext {
    // This is the only place where the API is web-sys specific and it should preferably be
    // used primarily from the Gloo canvas crate or something, not directly from the user.
    pub fn new(web_sys_context: web_sys::WebGlRenderingContext) -> Self
    {
        WebGlRenderingContext { context: web_sys_context }
    }
}

// Then it is 'just' implementing all of the webgl methods from web-sys. Let's start with a
// simple example.

impl WebGlRenderingContext {
    // Changing the input types to make sure that the method does not throw a runtime exception
    // is an easy win. No other errors can occur in this method so no need to return a result.
    pub fn viewport(&self, x: i32, y: i32, width: u32 /*changed from i32*/, height: u32/*changed from i32*/)
    {
        // Maybe check if the viewport has changed?
        self.context.viewport(x, y, width as i32, height as i32);
    }
}

// And another example:

#[derive(Debug, Eq, PartialEq)]
pub enum BlendType
{
    Zero,
    One,
    SrcColor,
    OneMinusSrcColor,
    SrcAlpha,
    OneMinusSrcAlpha,
    DstAlpha,
    OneMinusDstAlpha,
    ConstantColor,
    ConstantAlpha
    // ...
}

impl WebGlRenderingContext {
    // Again, the input parameter types makes the interface safe
    pub fn blend_func(&self, s_factor: BlendType, d_factor: BlendType) -> Result<(), Error>
    {
        // Check if ConstantColor and ConstantAlpha is used together (see https://developer.mozilla.org/en-US/docs/Web/API/WebGLRenderingContext/blendFunc)
        if s_factor == BlendType::ConstantColor && d_factor == BlendType::ConstantAlpha ||
            s_factor == BlendType::ConstantAlpha && d_factor == BlendType::ConstantColor
        {
            return Err(Error::WebGLError { message: "blend_func cannot be called with both ConstantColor and ConstantAlpha.".to_string() });
        }
        // Check if the blend state is already the desired blend state, if it is, then we don't call the webgl function!
        // ...
        Ok(())
    }
}

// So next example is a bit more complex and this involves exposing another API than the one in web-sys/WebGL.
// The goal here is not to copy the WebGL API, but rather to expose the exact same functionality as safely as possible.

pub struct VertexShader<'a>
{
    shader: web_sys::WebGlShader,
    context: &'a web_sys::WebGlRenderingContext
}

// Delete the shader after it has been used and possibly reused.
impl<'a> std::ops::Drop for VertexShader<'a>
{
    fn drop(&mut self)
    {
        self.context.delete_shader(Some(&self.shader));
    }
}

impl WebGlRenderingContext {
    pub fn create_vertex_shader(&self, source: &str) -> Result<VertexShader, Error>
    {
        let shader = self.context.create_shader(web_sys::WebGlRenderingContext::VERTEX_SHADER)
            .ok_or(Error::WebGLError {message: "Cannot create vertex shader!".to_string()})?;
        self.context.shader_source(&shader, source);
        self.context.compile_shader(&shader);

        if self.context.get_shader_parameter(&shader, web_sys::WebGlRenderingContext::COMPILE_STATUS).as_bool().unwrap_or(false)
        {
            Ok(VertexShader {shader, context: &self.context})
        } else {
            Err(Error::WebGLError { message: self.context.get_shader_info_log(&shader).unwrap_or_else(|| "Unknown error creating shader".into()) })
        }
    }

    pub fn create_program(&self, vertex_shader: VertexShader /*, fragment_shader: FragmentShader*/)
    {
        // ...
    }
}

#[derive(Debug)]
pub enum Error {
    WebGLError {message: String}
}

Drawbacks, rationale and alternatives

This crate solves the easy problems with the web-sys (and WebGL) interface (rusty safe interface). The more hard problems (simple to use, avoid state machine, performance etc.) is difficult to address in general and especially without creating an opinionated library. Therefore, I envision this crate to be the foundation for a multitude of different opinionated libraries as well as to be used by graphics programmers that want low level control.

The alternative, as I see it, is to go straight to the higher level crates and then let graphics programmers use the web-sys API. However, I would have appreciated a crate like this a few months ago and I am probably not the only one.

Unresolved questions

• IMO, the wrapper should be as safe and simple as possible, but not limit the programmer in any way. This means that you should be able to do exactly the same as with the web-sys API, and that might make the API a bit uglier, but how bad it is going to be, I don’t know yet. In the cases where it is inevitably going to be an ugly API, I want it to be as similar to the WebGL API as possible.
• Probably a lot more that I can’t think of right now…

[David-OConnor]

I like it, and agree that the center should be a WebGlRenderingContext wrapper. What do you think about storing configuration options along with it? Eg the things you can set with enable, as a start?

Here's one approach:

pub struct EnableCfg {
    blend: bool,
    cull_face: bool,
    depth_test: bool,
    dither: bool,
    // etc
}

impl Default for EnableCfg {
    fn default() -> Self {
        Self {
            dither: true,
            blend: false,
            // etc
        }
    }
}

pub struct WebGlRenderingContext
{
    context: web_sys::WebGlRenderingContext,
    enable_cfg: EnableCfg,
}

[grovesNL]

It seems fine for gloo to focus on WebGL only (i.e. instead of both OpenGL and WebGL), but it's also a bit restricting for existing crates which would like to target both OpenGL and WebGL.

For example, glium would need to have a separate WebGL backend which uses this new API, and gfx-hal would still need something like glow to map between native and web APIs (possibly still have to use web-sys directly).

I also think it would be better to leave state caching and validation out of the innermost layer, because this breaks concepts like GL context sharing for example.

[David-OConnor]

https://github.com/gfx-rs/gfx/pull/2554

[asny]

@David-OConnor Thanks! I like the idea og wrapping the states as well, which will allow for checking that the state actually changed before calling WebGL functions as well as expose debug info or in other ways help the programmer keep track of the states. As I said before, I think it is not possible to change it to something else than a state machine without creating a totally different API but this might be a nice addition which alligns well with the WebGL API. However, if it limits the functionality, I still think it is a no go. So does it limit functionality? Sharing ressources between WebGL contexts is not possible as far as I know. Anyone that can confirm this?

@grovesNL I am a big fan of what you are doing with glow, but for the basic crate I think it should be purely WebGL as there are differences between OpenGL and WebGL. Also gloo is about web utility crates as you also pointed out. That said, I see nothing wrong with creating a higher level crate that targets both WebGL and OpenGL and then it is up to the users decide which to use.

[David-OConnor]

I wonder if Gloo's role in this should be simple wrappers that improve the web_sys API, and deal with types better. (eg not expose JsValue, and cast things in ways bindgen can convert to JS typed arrays). There are multiple layers from raw [Web]Gl through full engines, and different APIs like WebGl, OpenGl, Vulkan etc that have mostly-overlapping functionality, with some differences. I think we should leave the mid/high-level things to dedicated, cross-API graphics crates like gfx, but provide a crate they can use to facilitate webgl-in-rust.

I've built a dual-API WebGl/Vulkano program, where the models, matrices etc were in Rust, and the WebGl part used #[wasm_bindgen]-wrapped fns that were called from JS, then handled the WebGl boilerplate there. A wrapper like this would have been a nice way to do this without JS.

[asny]

I think that is exactly the purpose of gloo, though I'm probably not the best to answer that question. And I agree that there are multiple layers from WebGL to a game engine and og course game engines are not suited for gloo, but a Glium like crate or regl.party like crate or similar might be.

[asny]

Continuing the discussion in this PR

[yoshuawuyts]

A guide to Rust Graphics Libraries in 2019 has been a very good read. In particular something they pointed at was the work currently being done on amethyst/rendy.

I don't think this would preclude any work on wrapping WebGL, but it seems like they're moving moving fast, and perhaps figuring out how to support them might be very valuable.

[grovesNL]

:+1: The crate I mentioned above (glow) is being used to implement gfx-hal WebGL support, and Rendy/Amethyst use gfx-hal.

[asny]

@grovesNL So you are definitely a potential user. Would you have used the suggested crate for glow if it was available? Or do you need something different?

@yoshuawuyts Really nice read indeed.

[grovesNL]

@asny For gfx-hal I was mostly looking for "raw" unified OpenGL/OpenGL ES/WebGL bindings with the same type signatures, which had common extension loading and version handling (optionally some really simple emulation like glMapBufferRange on WebGL). So basically what Emscripten would normally provide, but lighter weight.

For our use case, I'm not sure it's useful to have state caching and validation at this level. For example, we already have to track states internally. I think glium is probably the closest GL crate which provides validation, but it adds some overhead that we'd prefer to avoid.

[asny]

I think this is really interesting! I think anyone creating something like gfx-hal, Glium, vulkano etc would prefer low level access with no overhead. One option is therefore to create a really slim wrapper that avoids the JS types, adds some type safety and nothing else. That just seem like a lot og work for not a lot of gains and the potential users are very limited. Or what do you think? Another option is to create a more high level crate like Glium, vulkano etc. This would add a layer of safety that most hobby graphics programmers would prefer. I think this has a wider audience but might not fit into gloo? A third option is to create a utility library like rendy. I havent looked much into this but i like the idea that you can tale the easy option when you want and get low level access when you need that. I dont know how it could be implemented though.

[grovesNL]

I think anyone creating something like gfx-hal, Glium, vulkano etc would prefer low level access with no overhead.

Yeah I think this is true, although Vulkano is probably unlikely to ever have a WebGL backend because it targets Vulkan directly.

One option is therefore to create a really slim wrapper that avoids the JS types, adds some type safety and nothing else. That just seem like a lot og work for not a lot of gains and the potential users are very limited.

This is one of the goals of glow. Currently all JS types are only used internally, and the API only exposes primitives or its own types.

Another option is to create a more high level crate like Glium, vulkano etc. This would add a layer of safety that most hobby graphics programmers would prefer. I think this has a wider audience but might not fit into gloo?

It's definitely an option, but could be a pretty large effort based on the size of Glium. I think it would be interesting to have Glium use unified bindings (like glow), allowing it to add WebGL support without having to do much work (i.e. adding a new backend or flags). This would allow people already using Glium to get WebGL support pretty quickly.

A third option is to create a utility library like rendy. I havent looked much into this but i like the idea that you can tale the easy option when you want and get low level access when you need that. I dont know how it could be implemented though.

There could be some interesting ideas here – basically have some optional libraries built on top of low level bindings. I think some ideas might include: resource loading (async loading of images and then uploading them into GL textures), extension handling (checking whether functionality is enabled, fallbacks from WebGL2 to WebGL1 extensions), version parsing, state caching (built on top of the lower level bindings but otherwise like you described), ESSL shader reflection, etc. I already plan to add some kind of extension handling and version parsing in glow for example.

There might also be some good inspiration in WebGL libraries or engines, like regl, pex, three.js, the new abstraction in pixi.js, etc.

[asny]

Yeah I think this is true, although Vulkano is probably unlikely to ever have a WebGL backend because it targets Vulkan directly.

Yeah, I totally agree. What I meant was that Vulkano wants low level access to Vulkan, Glium to OpenGL etc.

It's definitely an option, but could be a pretty large effort based on the size of Glium. I think it would be interesting to have Glium use unified bindings (like glow), allowing it to add WebGL support without having to do much work (i.e. adding a new backend or flags). This would allow people already using Glium to get WebGL support pretty quickly.

I have to say that I am sceptical about Glium ever getting a WebGL backend, but you’ll never know. Also, I think this is part of a larger discussion whether to have a general API and multiple backends (like gfx and most game engines) or having an API which corresponds to a specific backend (like Glium, Vulkano etc.). One is not better than the other in my opinion, it very much depends on what you want to use it for. If you want to target all platforms and have limited resources, one API for multiple backends are definitely the way forward. But if you only want to target web, using a web specific API would probably save you some trouble down the road. Actually, I read a long discussion between Tomaka and Kvark about this (which I cannot find at the moment) and I don’t want to go down that road. Gloo is about web and it seems like the one-API-multiple-backends side is covered by for example yourself, so let’s focus on WebGL.

There could be some interesting ideas here – basically have some optional libraries built on top of low level bindings. I think some ideas might include: resource loading (async loading of images and then uploading them into GL textures), extension handling (checking whether functionality is enabled, fallbacks from WebGL2 to WebGL1 extensions), version parsing, state caching (built on top of the lower level bindings but otherwise like you described), ESSL shader reflection, etc. I already plan to add some kind of extension handling and version parsing in glow for example.

Yeah, I like it as well (actually, Three.js also do this, which I really liked)! Instead of incapsulating the web-sys rendering context, you always have access to it and are never limited by the API. This means that we can avoid handling edge cases and thereby expose a simpler interface. Also this will be helpful with only a limited part implemented since you can just use web-sys for the rest. Finally, this means complete independence between for example the program/shader setup and texture loading. The downside is that we cannot keep track of the states, since we never know if the user changed it directly through web-sys, but I think that is a small price to pay. I will do another proof-of-concept version of the API that reflects these thoughts, but it might take a while before I have time.

Can you elaborate a bit about the ideas behind version parsing and ESSL shader reflection?

There might also be some good inspiration in WebGL libraries or engines, like regl, pex, three.js, the new abstraction in pixi.js, etc.

I'll look more into them.

[dakom]

Wow, did not know this project existed! I was doing a similar thing as my first Rust project - and getting help from some of y'all in various issues along the way... cool! :)

One idea I'm playing with on paper (haven't coded it yet) and might be a good topic for discussion here is how to speed up uniform/attribute lookups.

Using a cache makes sense, but personally I wasn't able to get that to work on a practical level without wrapping some things in a RefCell. Might just be my newbie lack of Rust skills, but I put a considerable amount of effort and just couldn't avoid it.

Except now - I'm thinking that instead of populating the cache on first-lookup, populate it when the shader is compiled. It's a bit of a weird idea - to scan the shader code for uniform/attribute declarations, and then get those and set it in a local HashMap - but the win is that subsequent lookups avoid the overhead of setting the cache if it isn't set yet, and this should also solve my RefCell problem. Also it's fairly understandable for the shader compilation to be relatively expensive, so taking a hit there in order to save on every set_attribute_by_name() type call is ultimately a probable win.

This might be too opinionated to bake into gloo - but thought it might be worth mentioning since setting an attribute / uniform value by name is pretty fundamental and it would be great to have that be as fast as possible.

[asny]

Apparently, I am the only one answering so let me answer you as well :)

Nice project! It is indeed very similar to what this project is about, but I don’t think it has been settled yet how this project will turn out.

It is definitely nice to avoid retrieving the uniform location using webgl, but I am unsure how much of a performance impact it will actually do.

A way to do lazy construction in rust is to wrap your variable in an Option. In this case it is Option<HashMap<String, u32>> and then construct the hash map if the value is none. Nevertheless, I think in this case, it is much better to just construct the hash map right after compiling the shader to avoid lack when you already started drawing. You can even get a list of the uniform attributes using get_program_parameter with ACTIVE_UNIFORMS and get_active_uniform.

[dakom]

fwiw I've been continuing with that project mentioned above, and it's also been a great way to learn Rust...

Some barebones demos: https://awsm.netlify.com/ WebGL code: https://github.com/dakom/awsm/tree/master/src/webgl

The overall approach I took, after trying different things and hitting deadends, was to implement new traits on WebGlRenderingContext / WebGl2RenderingContext - and give the new functions the prefix awsm

So for example, depth_func is implemented as something kinda like:

pub trait WebGlFuncs {
    fn awsm_depth_func(&self, func:CmpFunction);
}

impl WebGlFuncs for WebGlRenderingContext {
    fn awsm_depth_func(&self, func:CmpFunction) {
        self.depth_func(func as u32);
    }
}

impl WebGlFuncs for WebGl2RenderingContext {
    fn awsm_depth_func(&self, func:CmpFunction) {
        self.depth_func(func as u32);
    }
}

And CmpFunction is an enum.

However, I don't really call those functions directly from the app side... there's a higher-level wrapper that stores the context, local lookups, etc. @asny - I did take your advice and cache all that stuff, on shader compilation :)

That higher level renderer then basically goes back to the normal names without the prefix, though being a higher level it doesn't usually match 1:1

Although I have wrapped a lot of the webgl stuff it's not automated and there's tons of stuff missing. I'm basically just adding stuff as I need it ;)

Not sure if this helps gloo directly in terms of API design, but hope there's some useful info here...