Asset Server

[stefnotch]

Asset Server flawed

So I wrote an asset server, but it turns out that an uncompressed Sponza scene file is really freaking heavyweight.

https://www.wolframalpha.com/input?i=297607031+bytes is how large it is when pointing bincode at an imported struct.

Which means that unless I whip out zero copy rkyv or https://github.com/Tommoa/rs-process-memory , the whole "transfer file over IPC and deserialize" approach is never going to work.

Which leads me to think...

Hot Reloading

With https://github.com/rksm/hot-lib-reloader-rs/tree/master , we could actually get game-engine-like behaviour. Where we keep the GPU buffers around during development, cause that's literally the fastest imaginable option.

We'd put "creating the Vulkan instance" into the main part, and everything else lives in the hot reloaded game. And the shared API would be something minimal like

• GetVk() -> VkSomething
• CacheBufferForNextReload(id, buffer)
• TryGetBuffer(id) -> buffer

Learned

Well, I still learned a lot from writing the asset server:

• I learned about Serde custom deserializers
  • serde json's json!() macro is fancy
  • serde json not handling Arc cycles is a lot less fancy
• I learned that deserializing is expensive and the rkyv is cool
• I learned how to use one specific interprocess communication library
• I learned more about error handling with Rust (thiserror and anyhow)
• I learned that cmake has "dependency files", which are super cool. That way, cmake does not need an extra pass to figure out which dependencies an input file actually has.
  • which leads to one really cool file watcher approach: save all file timestamps (+snapshot version) => compile => look at the dependencies file => load all the timestamps of the dependencies (+snapshot version) => compare all the file timestamps of the files that are written down in the dependencies file => if no changes, then the compilation was a huge success => else something changed during compilation, which is dangerous
  • btw filesystem timestamps are not monotonically increasing, so for change detection, one best compares them for equality and not for greater/less than
• I learned about JSON schema files. Turns out those are really neat.
• I got to do a lot of dark Rust sorcery using Any and downcasting and traits (trait X { fn as_any(&self) -> Any } is a funni trick)
• I learned about redb and sled (embedded databases)
• I did a bit of logging, turns out Rust has standardized error, warning, info, ... logging facilities.

also

• I also learned that cargo has build dependencies
• I looked at a bit of Rust multi threading stuff