Automatically build shaders

[ian-h-chamberlain]

Similar to the cc-rs crate, it would be nice to compile shaders automatically in one of these ways:

• build script helper crate (probably looks very similar to cc-rs in practice)

  • include_bytes!(concat!(env!("OUT_DIR"), "/shader.shbin")) ? looks kinda messy but would work

  • somehow include built shader bin in romfs – maybe hard link / symlink can do this?

• Proc macro? Basically would run the shader compiler and spit out bytes... probably would increase compile time poorly

[Meziu]

There is the ancient https://github.com/rust3ds/picasso-rs crate to solve this issue. I don't know if it's up to date, but it's a possible way. In the case we can't use it or we want to remake it I'll archive it.

[ian-h-chamberlain]

Yeah, unfortunately, I looked at the code there and it's not much different than what the build script here is doing.

I suppose having it as a library would make it easier for others to use in their own code, but would still depend on the picasso executable (which I doubt we want to rewrite from scratch, at least not yet).

Let's keep the picasso-rs repo around for now, it could be worth reviving even from its current form. Ideally I'd like a way to just call something like const COMPILED_SHADER = picasso::shader!("xyz.pica"); but it might take a bit of work to get there.

[adryzz]

Tried writing a proc macro that takes a pica file and using the picasso tool it compiles it and include_bytes! it, but OUT_DIR is not defined for proc macros, so this cannot work, as i have nowhere to put the compiled shader.

One way i found (and that i'm trying to work on) is compiling picasso as a library, and using that inside of the proc macro to directly output the compiled shader.

One other advantage of this is that it additionally allows for writing shaders inline as well.

[ian-h-chamberlain]

Tried writing a proc macro that takes a pica file and using the picasso tool it compiles it and include_bytes! it, but OUT_DIR is not defined for proc macros, so this cannot work, as i have nowhere to put the compiled shader.

Hmm, I wonder... could the proc-macro read the bytes and emit them directly (instead of emitting an include_bytes!() call)? The actual file output could be emitted to a temporary file/dir, I'm thinking something like this:

use tempdir::TempDir; // https://docs.rs/tempdir/latest/tempdir/

let tmp_dir = TempDir::new("picasso").unwrap();
let file_path = tmp_dir.path().join("out.shbin");

let status = process::Command::new("picasso")
    .args(["-o", file_path, "src.pica"])
    .status()
    .unwrap();

if !status.success() {
    panic!("failed")
}

let bytes = std::fs::read(file_path).unwrap();
// emits a token stream like `[10u8, 11u8, ... ]`
quote! {
    [ #(#bytes),* ]
}

That definitely has the downside of having to recompile the shader every time, but maybe with some kind of caching this could be avoided. Or maybe that behavior is actually preferable, if the shader assembly is fairly quick.

---

One way i found (and that i'm trying to work on) is compiling picasso as a library, and using that inside of the proc macro to directly output the compiled shader.

Huh, that's definitely an interesting idea! It doesn't look like it was written to be used as a library so I imagine you'll probably have to reimplement a good chunk of the frontend, but this definitely seems like the more Rust way to do it, if it's feasible. I'm not sure how this library would be packaged, since I assume the sources are only available on github and would need to be compiled a specific way to be linked into the proc-macro. Still, it definitely seems worth exploring to me.

[adryzz]

The actual file output could be emitted to a temporary file/dir, I'm thinking something like this:

this cannot be done. cargo refuses to publish crates with build scripts/macros that output stuff anywhere outside OUT_DIR (which is a good thing tbh). And, as we said before, OUT_DIR isn't available for macros (and there are no plans to do that)

We could redirect the file to stdout, but that just makes it non-portable and bad

as for the building picasso as a library, while that can be done, rust-bindgen doesn't support the C++ features that picasso uses, and also picasso uses autogen which is a mess for building stuff in build scripts anyway

The only other option is to rewrite picasso in rust, and i'm trying to do that. it's not super hard to do, but the code is definitely a mess because it uses static variables, and static mut in rust is a no-no.

Additionally, it uses C strings, and its type safety isn't that great (it often uses u64s in functions that accept u32s, mixes up signed and unsigned integers...) and all that stuff just doesn't fly with rust, so i have to make everything explicit.

Anyway, making slow progress here. The macro does work already, the shader assembler not yet, so while it does generate a valid binary, it's currently empty, and only has the headers and stuff.

The code is a big mess, and i plan to refactor it in its entirety to rust standards once i'm done porting all the C++ code

[ian-h-chamberlain]

this cannot be done. cargo refuses to publish crates with build scripts/macros that output stuff anywhere outside OUT_DIR (which is a good thing tbh). And, as we said before, OUT_DIR isn't available for macros (and there are no plans to do that)

Hmm, I guess it's surprising that a macro can't use temporary directories, but I can see the reasoning for not wanting to allow writing to arbitrary filesystem locations. Do you know of somewhere that rule is documented? The only thing I could find was related but not quite the same (rule 4 of https://doc.rust-lang.org/cargo/commands/cargo-package.html#description).

I did find https://stackoverflow.com/a/56479446/14436105 which might provide a workaround of sorts... It is kinda similar to just using OUT_DIR in a build script, and I think sidesteps the requirements you mentioned.

That said, I commend your efforts in porting the assembler to Rust and look forward to seeing progress!

[0x00002a]

So I've got a basic rust version of picasso I've called pican (repo). It supports enough of picasso's assembly to compile the mesh shader we used in bevy 3ds. I wonder if there would be support for integrating this as an optional alternative to picasso in the include macro. The main thing pican needs right now is more test data, it supports enough to write I think anything (as long as its not geo shaders) but I don't have any actual real assembly to test the features it doesn't support yet.

Additionally its IR is quite rich and it has support for disassembling the shader blobs into a nicer format than the shbin one in ctru-rs so I'm hoping it can be used to improve validation on things like uniform bindings as well