Open fogti opened 2 years ago
As for dealing with NARs the interface in Nix.rs is event driven so it can deal with NARs of any size without having to store the whole NAR in memory. It is also complete in that has readers and writers for both NAR files and filesystem and is fully async.
oh, I forgot to mention another thing which is especially important for the builder backend: it needs to be able to bind-mount stuff. although that might be an detail which doesn't need to be represented in a general interface because usually stuff just gets copied beforehand.
I also added https://github.com/YZITE/yzix/blob/c19f56d36d14eec62e7d0daea873ade8b477fc38/crates/yzix-core/src/ser_trait.rs to yzix
to be able to change the network serialization framework later without affecting hashes. Only disadvantage is that it isn't async
... But adding streaming support should be realistic.
Regarding store abstractions, it would be nice to know / extrapolate what exactly is necessary for the interpreter side to deal with stores. Thus, I want to sort-of collect stuff here which I think may be important (it if gets too off-topic, I'll move stuff to https://github.com/RIIR-Nix/rfcs instead) Also, the operations of the store can be split into three categories: Reading, Writing, Building. I think it might be a good idea to model this via multiple (maybe
async
) traits, especially because some stores don't support Writing (read-only) or Building (S3), e.g.(derivation building is a bit weird, because the derivation function takes an attrset containing a pretty wild mix, and doesn't explicitly separate what ends up in the environment, what doesn't, might contain magic modifiers (
__contentAddressed
,__structuredAttrs
,...), etc. (see also, somewhat generic abstraction of that))Another question would be how to deal with store paths, store hashes and potentially content addressing, and the three different "types"/kinds of store paths/hashes:
Also, store paths have some sort of common format, which is useful when e.g. rewriting store paths in dumps or such (pretty much implemented here (although that store path format deviates a bit from the nix format, which is the reason why
store_ref_scanner::StoreSpec
exists))Another thing to abstract are "store dumps" (e.g. deserialized NARs), e.g. something like https://github.com/YZITE/yzix/blob/cdd10d1249d6b6d8a79ed04d6acb01f92c932117/crates/yzix-core/src/dump.rs#L13-L17