nannou-org / nannou

A Creative Coding Framework for Rust.
https://nannou.cc/
6.04k stars 305 forks source link

Appification / Final Push #969

Closed tychedelia closed 3 months ago

tychedelia commented 6 months ago

This code is probably going to be pretty impossible to review in the diff. Highly suggest checking it out and just playing around with it.

Changes

Draw materials

The Draw instance is now generic over a bevy material. By default, we expose a DefaultNannouMaterialthat is what most users will interact with. This material is a material extension over the bevy StandardMaterial, which is their default PBR material and contains a variety of useful properties that users can now use.

Rendering materials

The rendering algorithm is significantly complicated by making Draw generic. Every time a user mutates their drawing in a way that requires a new material, this requires storing that material instance within our draw state. Further, we support changing that material's type (as described below) for any given draw instance or drawing, which transitions the type of the draw instance from Draw<M> to Draw<M2>. Because these cloned / mutated draw instances still point to the same "logical" draw, we need a way to type erase materials within our draw state.

This is accomplished in the following manner:

Custom materials

A variety of new possibilities are opened up by allowing users to create their own materials. The bevy Material allows easy access to writing a vertex or fragment shader for a given drawing, which means user's can easily include any arbitrary data they want when writing a custom shader for a given drawing primitive. What this means in practice is that rather than writing manual wgpu code, the user will write something like the following in order to render a fullscreen shader:

let win = app.window_rect();
    draw
        .material(CustomMaterial::default())
        .rect()
       .x_y(win.x(), win.y());

Removal of Texture drawing primitive

Previously, a texture was rendered in nannou using a special primitive that instructed the fragment shader to sample a given texture. This has primitive has been removed. Users now should use the material method texture that sets the bevy material's texture to a given Handle<Image>, which represents an asset handle to a wgpu texture. Similarly to above, this means that you now will render a simple texture as a `rect:

use nannou::prelude::*;

fn main() {
    nannou::app(model).run();
}

struct Model {
    texture: Handle<Image>,
}

fn model(app: &App) -> Model {
    app.new_window().size(512, 512).view(view).build();
    let texture = app.assets().load("images/nature/nature_1.jpg");
    Model { texture }
}

fn view(app: &App, model: &Model) {
    let draw = app.draw();
    draw.background().color(BLACK);
    let win = app.window_rect();
    draw.rect().x_y(win.x(), win.y()).texture(&model.texture);
}

In the model function, we now use bevy's AssetLoader exposed via the world in order to load images as an asset handle. These handles can then be used to load an image into cpu memory to do things like configure the given sampler for an image. In general, this provides a lot more flexibility and should allow users to do more powerful things with textures without having to drop into our "points" API.

Mesh UVs

In order to render textures and do anything interesting with custom shaders, it is now important that every drawing primitive has texture coordinates, and these are a required field in our vertex layout. Drawing primitives have been changed to provide a set of default UVs where sensible. Additionally, a points_vertex method has been added to our "points" API that allows the user to provide explicit UVs in additional to position and color data. Our "full" vertex type is now always (Vec2, Color, Vec2).

Additionally, a new SetTexCoords property trait has been added that allows users to change the UVs for a given drawing primitive. For example, our rect has a sensible [0.0, 0.0] to [1.0, 1.0] set by default, but users may want to sample from a smaller part of a texture:

let area = geom::Rect::from_x_y_w_h(0.5, 0.5, 1.0, 1.0);
draw.rect()
            .x_y(x2, y2)
            .w_h(w, h)
            .texture(&texture)
            .area(area);

The area method (final name tbd) here of SetTexCoords allows providing a geom::Rect that updates the UVs. In this way, while we make a best effort to provide sensible default values, users can override those values which may be particularly useful for things like triangles or circles, etc.

There are still a few paths where we may fall back to a default Vec2::ZERO, but this can hopefully be eliminated over time. We also still provide the points_colored API for instances where users truly don't care about their texture coordinates, but anyone working with textures or custom shaders will need to provide their own.

App changes

Our App instance now wraps bevy's world as a Rc<RefCell<UnsafeWorldCell<'w>>>. This interior mutability allows us to access potentially mutating methods on the bevy world without having to use &mut Self on our App instance.

App methods

All of App's methods now borrow our internal reference to world and return that data to a user. Sometimes, this means returning an instance of a bevy resource directly (e.g. Time or AssetLoader). Due to the use of interior mutability, it's theoretically possible for the user to cause a panic, although I have not experienced that yet. We'll want to be careful and watch to ensure that this isn't easy to do.

Creating App in internal systems

Because we store our nannou bookkeeping state (i.e. windows, event handler function pointers, the user's model) in the bevy world, there's a bit of unsafety and complexity around how to then provide an instance of App to the user that wraps that world.

Each system that runs a given event handler declares the dependencies it requires to be extracted from the world before creating the App instance to pass to the user's handler. For example:

#[allow(clippy::type_complexity)]
fn key_events<M>(
    world: &mut World,
    state: &mut SystemState<(
        EventReader<KeyboardInput>,
        Query<&WindowUserFunctions<M>>,
        NonSendMut<M>,
    )>,
) where
    M: 'static

This SystemState is then used to unsafely extract the state and create an App instance:

fn get_app_and_state<'w, 's, S: SystemParam + 'static>(
    world: &'w mut World,
    state: &'s mut SystemState<S>,
) -> (App<'w>, <S as SystemParam>::Item<'w, 's>) {
    state.update_archetypes(world);
    let app = App::new(world);
    let param = unsafe { state.get_unchecked_manual(*app.world.borrow_mut()) };
    (app, param)
}

This is dangerous! The primary invariant we must uphold here is that we never expose methods on App that would allow mutable reference to the state we extract from world here. In many cases this is fine, because we are extracting internal nannou state that user's don't need to know about, but is a potential source of soundness issues.

TODO:

~- [ ] Figure out how to interlace draw cmds where primitives may mutate their own material.~