boulder: binary manifests don't list the files a build will generate?

[ermo]

Inspecting a manifest.jsonc reveals a list of filenames w/full paths that a recipe build will generate.

When I do a moss inspect manifest.x86_64.bin, no such list of files is printed? Why is that?

[tarkah]

We don't add content / layout / index payloads, those are for binary stones.

[ermo]

Just capturing the discussion notes here for reference:

Ikey Doherty JsonC provides a human relevant diff for us for the files So we can easily review for weirdness and know whats in it at a glance Its never held any interest for the build automation as a .bin is just a series of augmented meta payloads, no layout payloads tarkah Yeah I just commented on the issue, content / index / layout payloads only go to the binary stone ermo if a manifest is a promise, why don't we care what files we promise to create? consider the case of a source-only index Ikey Doherty Because we only care about providers for planning Those are expressed in meta payload Ie pkgconfig etc ermo you want to know which recipe provides /usr/bin/proprietary-thing tarkah Right providers is the guarantee, not the list of files. Things can't depend on an explicit file path in recipe Ikey Doherty binary() Solved problem :) ermo ok, /usr/share/man-page Ikey Doherty Right now nothing cares for it If there's no way to depend on it then there's no way to express it A separate problem is "how do I locate files for a repo?" Which would be a files index on a binary repo, which isn't a solved issue As the original vision was melding the indices to augment in both directions ermo specifically, a from-source-only repo in this case, i.e. a repo with no associated .stones stored in it (only manifests) Ikey Doherty I get what you're saying, you should be able to use whatis/whereis ermo but as manifests are supposed to act as "promises" (in some fashion), then why not capture the filenames a manifest will produce? No other metadata than that, mind. Ikey Doherty And know what source yields a thing ermo yes, exactly. Ikey Doherty Problem is we're not Sabayon The manifest format is deliberately simple and small Augment recipe with build proof and enough provider information to plan builds Ie power summit or source builds Chicken and egg isn't solvable as someone needs to bootstrap first to produce the manifests That was the compromise Tldr without something like Bootstrap Bill we're not capable of being purely source based ie, you can't know what you don't know ermo ... Bootstrap Bill is that tool you envisioned could rebuild/reboostrap a repo? (name pulled from Pirates of the Carribean I believe?) Ikey Doherty The issue here is one of evolutionary levels A basic bootstrap requires explicit ordering, easy enough, and name based deps, audited by humans As they can't know in advance the yield Our current operating layer is higher, where we have a bootstrapped system that can produce these unpredictable yields Recording as manifests ermo Can we make a distinction on required fields and allowed fields in manifest.bin or is that still too "much"? meaning manifests may provide filenames yield, but does not require it, for the reasons you just outlined Ikey Doherty The issue is they're different payloads How do you then know which layout belongs to what ? Without introducing a container type into the format Even at bootstrap level the files themselves don't exactly matter either ermo so it's also an architectural/format issue I suppose. i.e. it has wider implications. Ikey Doherty Its more about chained builds and dep tiers Step 1 producing a bootstrap is deviation right? So you produce a temporary toolchain for your target That's a deviation from the stage3 we ship with So it's impossible to define in the packaging model unless you have two separate recipes (This is achieved in rpm land with conditionals in a single recipe) Also at some point I'd love if we all agree that packaging libraries makes no sense And that the repo should be a way to produce built apps. Not a tapestry exercise ^^ ermo I think what I'm saying is "how is it that, on a practical level, we can add files just fine to the .jsonc manifest after building, but adding them to the binary manifest after building is not possible, even as optional payloads that only contain the filenames like in .jsonc"? Ikey Doherty Because we only write jsonc Boulder never reads it It's basically perl Keep in mind we have no concept of optional payloads when it comes to streaming a binary archive If it doesn't % 2 then .. do what ? And the only mechanism we have for recording files in a format legal way is the layout payload Composed of LayoutEntry in D speak There's no record tag for a file path to go into meta And it'd seriously inflate the sizes tbf ermo so the sticking point is "the format legal way". And we can't change that yet. OK. (and whether we want to is also a different discussion) for the from-source side to be "good", this will probably need to be considered. If we have bigger fish to fry right now, ok. Ikey Doherty The tldr is currently we can't record the files as binary We could, if we had a strong enough use case Which entails the whole figure out bootstrap thing ermo I don't think it's important enough until we are ready to really look at the from-source use case, if push comes to shove. but it will probably need figuring out at some point within, say, the next year or so (probably >6 months at the very least) so for now, I'm mentally shelving it FWIW. as it's more of a nice "what if", than a "we need this yesterday". Ikey Doherty We know we can augment and bump