This code is probably going to be pretty impossible to review in the diff. Highly suggest checking it out and just playing around with it.
Changes
Draw materials
The Draw instance is now generic over a bevymaterial. By default, we expose a DefaultNannouMaterialthat is what most users will interact with. This material is a material extension over the bevyStandardMaterial, which is their default PBR material and contains a variety of useful properties that users can now use.
Rendering materials
The rendering algorithm is significantly complicated by making Draw generic. Every time a user mutates their drawing in a way that requires a new material, this requires storing that material instance within our draw state. Further, we support changing that material's type (as described below) for any given draw instance or drawing, which transitions the type of the draw instance from Draw<M> to Draw<M2>. Because these cloned / mutated draw instances still point to the same "logical" draw, we need a way to type erase materials within our draw state.
This is accomplished in the following manner:
We store a last_material within state that tracks the id of the last known material, which works similarly to tracking context.
Every time a new drawing is started, a None slot is added to draw_commands which can be used if the material is later changed for that drawing.
If last_material has changed, we push a DrawCommand::Material(UntypedAssetId). Here, UntypedAssetId is the mechanism through which we achieve type erasure. Inside the draw state, we store a HashMap<UntypedAssetId, Box<dyn Any + Send + Sync>>, which contains all the materials that are used in our drawing and will be downcast later when processed for rendering.
Inside an individual Drawing, we now store a copy on write pointer type DrawRef that can either be a reference to a parent draw instance OR a owned clone of that draw instance with a different material parameter. The state of ref is used to determine on drop whether we mutated the material in our drawing, and thus whether a new material needs to be inserted into draw_commands at the index slot we created in advance when the drawing was created.
In a separate bevy system that runs after our drawing, we iterate through the materials stored in State and try to downcast them into a given registered material type. We then add the material as an asset to the bevy world so it can be used by our mesh rendering system.
When rendering a drawing, we iterate through the stack of draw commands and use our material command as a signal to render into a new mesh. What this means in practice is that if a user never switches materials by calling a mutating material method, their entire drawing will render into the same mesh. In other words, the number of meshes == the number of materials.
Custom materials
A variety of new possibilities are opened up by allowing users to create their own materials. The bevyMaterial allows easy access to writing a vertex or fragment shader for a given drawing, which means user's can easily include any arbitrary data they want when writing a custom shader for a given drawing primitive. What this means in practice is that rather than writing manual wgpu code, the user will write something like the following in order to render a fullscreen shader:
let win = app.window_rect();
draw
.material(CustomMaterial::default())
.rect()
.x_y(win.x(), win.y());
Removal of Texture drawing primitive
Previously, a texture was rendered in nannou using a special primitive that instructed the fragment shader to sample a given texture. This has primitive has been removed. Users now should use the material method texture that sets the bevy material's texture to a given Handle<Image>, which represents an asset handle to a wgpu texture. Similarly to above, this means that you now will render a simple texture as a `rect:
use nannou::prelude::*;
fn main() {
nannou::app(model).run();
}
struct Model {
texture: Handle<Image>,
}
fn model(app: &App) -> Model {
app.new_window().size(512, 512).view(view).build();
let texture = app.assets().load("images/nature/nature_1.jpg");
Model { texture }
}
fn view(app: &App, model: &Model) {
let draw = app.draw();
draw.background().color(BLACK);
let win = app.window_rect();
draw.rect().x_y(win.x(), win.y()).texture(&model.texture);
}
In the model function, we now use bevy's AssetLoader exposed via the world in order to load images as an asset handle. These handles can then be used to load an image into cpu memory to do things like configure the given sampler for an image. In general, this provides a lot more flexibility and should allow users to do more powerful things with textures without having to drop into our "points" API.
Mesh UVs
In order to render textures and do anything interesting with custom shaders, it is now important that every drawing primitive has texture coordinates, and these are a required field in our vertex layout. Drawing primitives have been changed to provide a set of default UVs where sensible. Additionally, a points_vertex method has been added to our "points" API that allows the user to provide explicit UVs in additional to position and color data. Our "full" vertex type is now always (Vec2, Color, Vec2).
Additionally, a new SetTexCoords property trait has been added that allows users to change the UVs for a given drawing primitive. For example, our rect has a sensible [0.0, 0.0] to [1.0, 1.0] set by default, but users may want to sample from a smaller part of a texture:
let area = geom::Rect::from_x_y_w_h(0.5, 0.5, 1.0, 1.0);
draw.rect()
.x_y(x2, y2)
.w_h(w, h)
.texture(&texture)
.area(area);
The area method (final name tbd) here of SetTexCoords allows providing a geom::Rect that updates the UVs. In this way, while we make a best effort to provide sensible default values, users can override those values which may be particularly useful for things like triangles or circles, etc.
There are still a few paths where we may fall back to a default Vec2::ZERO, but this can hopefully be eliminated over time. We also still provide the points_colored API for instances where users truly don't care about their texture coordinates, but anyone working with textures or custom shaders will need to provide their own.
App changes
Our App instance now wraps bevy's world as a Rc<RefCell<UnsafeWorldCell<'w>>>. This interior mutability allows us to access potentially mutating methods on the bevy world without having to use &mut Self on our App instance.
App methods
All of App's methods now borrow our internal reference to world and return that data to a user. Sometimes, this means returning an instance of a bevy resource directly (e.g. Time or AssetLoader). Due to the use of interior mutability, it's theoretically possible for the user to cause a panic, although I have not experienced that yet. We'll want to be careful and watch to ensure that this isn't easy to do.
Creating App in internal systems
Because we store our nannou bookkeeping state (i.e. windows, event handler function pointers, the user's model) in the bevy world, there's a bit of unsafety and complexity around how to then provide an instance of App to the user that wraps that world.
Each system that runs a given event handler declares the dependencies it requires to be extracted from the world before creating the App instance to pass to the user's handler. For example:
This is dangerous! The primary invariant we must uphold here is that we never expose methods on App that would allow mutable reference to the state we extract from world here. In many cases this is fine, because we are extracting internal nannou state that user's don't need to know about, but is a potential source of soundness issues.
TODO:
~- [ ] Figure out how to interlace draw cmds where primitives may mutate their own material.~
Actually this is okay. If a drawing changes it's material, we probably do want it to have a higher z-value as well as reset the mesh/material of its parents. I think this behavior makes more sense atm.
[x] Add all material methods.
[ ] Ensure UVs are correctly calculated for all primitives.
This code is probably going to be pretty impossible to review in the diff. Highly suggest checking it out and just playing around with it.
Changes
Draw materials
The
Draw
instance is now generic over abevy
material. By default, we expose aDefaultNannouMaterial
that is what most users will interact with. This material is a material extension over thebevy
StandardMaterial
, which is their default PBR material and contains a variety of useful properties that users can now use.Rendering materials
The rendering algorithm is significantly complicated by making
Draw
generic. Every time a user mutates their drawing in a way that requires a new material, this requires storing that material instance within our draw state. Further, we support changing that material's type (as described below) for any given draw instance or drawing, which transitions the type of the draw instance fromDraw<M>
toDraw<M2>
. Because these cloned / mutated draw instances still point to the same "logical" draw, we need a way to type erase materials within our draw state.This is accomplished in the following manner:
last_material
within state that tracks the id of the last known material, which works similarly to tracking context.None
slot is added todraw_commands
which can be used if the material is later changed for that drawing.last_material
has changed, we push aDrawCommand::Material(UntypedAssetId)
. Here,UntypedAssetId
is the mechanism through which we achieve type erasure. Inside the draw state, we store aHashMap<UntypedAssetId, Box<dyn Any + Send + Sync>>
, which contains all the materials that are used in our drawing and will be downcast later when processed for rendering.Drawing
, we now store a copy on write pointer typeDrawRef
that can either be a reference to a parent draw instance OR a owned clone of that draw instance with a different material parameter. The state of ref is used to determine on drop whether we mutated the material in our drawing, and thus whether a new material needs to be inserted intodraw_commands
at the index slot we created in advance when the drawing was created.bevy
system that runs after our drawing, we iterate through the materials stored inState
and try to downcast them into a given registered material type. We then add the material as an asset to thebevy
world so it can be used by our mesh rendering system.Custom materials
A variety of new possibilities are opened up by allowing users to create their own materials. The
bevy
Material
allows easy access to writing a vertex or fragment shader for a given drawing, which means user's can easily include any arbitrary data they want when writing a custom shader for a given drawing primitive. What this means in practice is that rather than writing manualwgpu
code, the user will write something like the following in order to render a fullscreen shader:Removal of
Texture
drawing primitivePreviously, a texture was rendered in
nannou
using a special primitive that instructed the fragment shader to sample a given texture. This has primitive has been removed. Users now should use the material methodtexture
that sets thebevy
material's texture to a givenHandle<Image>
, which represents an asset handle to awgpu
texture. Similarly to above, this means that you now will render a simple texture as a `rect:In the
model
function, we now usebevy
'sAssetLoader
exposed via the world in order to load images as an asset handle. These handles can then be used to load an image into cpu memory to do things like configure the given sampler for an image. In general, this provides a lot more flexibility and should allow users to do more powerful things with textures without having to drop into our "points" API.Mesh UVs
In order to render textures and do anything interesting with custom shaders, it is now important that every drawing primitive has texture coordinates, and these are a required field in our vertex layout. Drawing primitives have been changed to provide a set of default UVs where sensible. Additionally, a
points_vertex
method has been added to our "points" API that allows the user to provide explicit UVs in additional to position and color data. Our "full" vertex type is now always(Vec2, Color, Vec2)
.Additionally, a new
SetTexCoords
property trait has been added that allows users to change the UVs for a given drawing primitive. For example, ourrect
has a sensible[0.0, 0.0]
to[1.0, 1.0]
set by default, but users may want to sample from a smaller part of a texture:The
area
method (final name tbd) here ofSetTexCoords
allows providing ageom::Rect
that updates the UVs. In this way, while we make a best effort to provide sensible default values, users can override those values which may be particularly useful for things like triangles or circles, etc.There are still a few paths where we may fall back to a default
Vec2::ZERO
, but this can hopefully be eliminated over time. We also still provide thepoints_colored
API for instances where users truly don't care about their texture coordinates, but anyone working with textures or custom shaders will need to provide their own.App changes
Our
App
instance now wrapsbevy
's world as aRc<RefCell<UnsafeWorldCell<'w>>>
. This interior mutability allows us to access potentially mutating methods on thebevy
world without having to use&mut Self
on ourApp
instance.App
methodsAll of
App
's methods now borrow our internal reference to world and return that data to a user. Sometimes, this means returning an instance of abevy
resource directly (e.g.Time
orAssetLoader
). Due to the use of interior mutability, it's theoretically possible for the user to cause a panic, although I have not experienced that yet. We'll want to be careful and watch to ensure that this isn't easy to do.Creating
App
in internal systemsBecause we store our
nannou
bookkeeping state (i.e. windows, event handler function pointers, the user's model) in thebevy
world, there's a bit of unsafety and complexity around how to then provide an instance ofApp
to the user that wraps that world.Each system that runs a given event handler declares the dependencies it requires to be extracted from the world before creating the
App
instance to pass to the user's handler. For example:This
SystemState
is then used to unsafely extract the state and create anApp
instance:This is dangerous! The primary invariant we must uphold here is that we never expose methods on
App
that would allow mutable reference to the state we extract from world here. In many cases this is fine, because we are extracting internalnannou
state that user's don't need to know about, but is a potential source of soundness issues.TODO:
~- [ ] Figure out how to interlace draw cmds where primitives may mutate their own material.~
UpdateMode
examples.