The recommended workflow for aggregation (supporting using cargo vet in multiple repositories) is to run the command cargo vet aggregate sources.list > audits.toml. This will wipe audits.toml and create a new audits file with the current content of all the sources. I find this impractical.
I have an organization with multiple repositories. Say we have org/product-a, org/product-b and org/cool-experiment. We then create an aggregated repository at org/supply-chain and aggregate the three. The team working on cool-experiment put a lot of time into auditing foo-compute and bar-visualize and add their audits to their local audits.toml. It gets aggregated, cool.
A while later this project is cancelled or sold to another company or just handed over to another OSS maintainer. We can now no longer guarantee the availability of org/cool-experiment/supply-chain/audits.toml. And even if it continues to exist under another organization, we no longer want to blindly trust new additions to this file! But we still want to keep on record that we have audited foo-compute and bar-visualize. Maybe org/product-a and/or org/product-b start using these dependencies, and they should of course not need to audit them again, since trusted people in the org already did. How to handle this?
Repeat the audits whenever needed in another repository - A lot of wasted time, but feels like the only solution the tooling has first class support for?
Manually copy the audits into product-a and product-b's own audits.toml. Causes repetition if it has to be done to both. Also, not handy if we don't use foo-compute today, but might in a year. When is it a good time to "vendor" this audit entry?
Stop using the aggregate functionality and instead manually curate the shared org/supply-chain/audits.toml with all audits done throughout the organization. I currently find this the most solid approach. But since there is zero tooling support for it, it does not feel like how cargo vet is supposed to be used.
Is there a recommended workflow I'm missing here?
Same problem for third party audits
I have a very similar issue with imported third party audits. And I'll bring it up here since it's so related. This is related to #601. Let's say I'm responsible for security for cool-experiment at the org above. cool-experiment pulls in the log dependency. cargo vet screams at me/the developer. I go look in the global registry and I find that Mozilla made an audit for the version of log we want. I evaluate the note attached to the audit and the trustworthiness of the person who signed off on the audit. I decide to approve it. With the tooling available it would now end up in imports.lock. But with the downside the tool now pulls in everything from Mozilla as fully trusted. I instead vendor this audit to cool-experiment/supply-chain/audits.toml maybe. Doesn't matter for the upcoming problem.
A while later one of product-a or product-b want to pull in the log dependency. They now have the same problem again. They could resolve it way quicker if they could easily see that I already found a trustworthy audit for this crate, and they can rely on my judgement without having to evaluate third parties at all. But there is no tooling for this in cargo vet as far as I can see. Maybe I'm responsible for security for all three repos. But I can't remember every crate I ever audited. Do I need to scout around in all our repositories to see if I have already evaluated this dependency or not?
Best solution I have come up with so far
What I'd like is a persistent record of all third party audits someone in my organization has decided to approve. And one list of all audits performed within the organization. All projects within the organization can just trust these audits without looking twice. And the latter list (our own audits) can be published on the registry.
Currently the best way for this I have come up with is to set up a org/supply-chain repository like this:
audits-3rd-party.toml - Manually curated list of approved third party audits, with aggregated-from field to show attribution.
audits.toml - Manually curated list of all audits performed by all teams at org. This can be published to the public registry for consumption by the entire world. This would be append only (unless we realize we made a mistake).
~audits-unified.toml - Created by cargo vet aggregate sources.list where sources.list point to the two files above ^. Combining all internal and third party audits that anyone in our org can rely on. This is the file all our code repositories point to in their import statements.~ EDIT: This can be skipped. All the downstream repositories can probably just import both above files. Fewer changes to commit to the supply-chain repo that way.
And then configure all our products like this (in product-x/supply-chain/config.toml):
The recommended workflow for aggregation (supporting using cargo vet in multiple repositories) is to run the command
cargo vet aggregate sources.list > audits.toml. This will wipeaudits.tomland create a new audits file with the current content of all the sources. I find this impractical.I have an organization with multiple repositories. Say we have
org/product-a,org/product-bandorg/cool-experiment. We then create an aggregated repository atorg/supply-chainand aggregate the three. The team working oncool-experimentput a lot of time into auditingfoo-computeandbar-visualizeand add their audits to their localaudits.toml. It gets aggregated, cool.A while later this project is cancelled or sold to another company or just handed over to another OSS maintainer. We can now no longer guarantee the availability of
org/cool-experiment/supply-chain/audits.toml. And even if it continues to exist under another organization, we no longer want to blindly trust new additions to this file! But we still want to keep on record that we have auditedfoo-computeandbar-visualize. Maybeorg/product-aand/ororg/product-bstart using these dependencies, and they should of course not need to audit them again, since trusted people in the org already did. How to handle this?product-aandproduct-b's ownaudits.toml. Causes repetition if it has to be done to both. Also, not handy if we don't usefoo-computetoday, but might in a year. When is it a good time to "vendor" this audit entry?org/supply-chain/audits.tomlwith all audits done throughout the organization. I currently find this the most solid approach. But since there is zero tooling support for it, it does not feel like howcargo vetis supposed to be used.Is there a recommended workflow I'm missing here?
Same problem for third party audits
I have a very similar issue with imported third party audits. And I'll bring it up here since it's so related. This is related to #601. Let's say I'm responsible for security for
cool-experimentat the org above.cool-experimentpulls in thelogdependency.cargo vetscreams at me/the developer. I go look in the global registry and I find that Mozilla made an audit for the version oflogwe want. I evaluate the note attached to the audit and the trustworthiness of the person who signed off on the audit. I decide to approve it. With the tooling available it would now end up inimports.lock. But with the downside the tool now pulls in everything from Mozilla as fully trusted. I instead vendor this audit tocool-experiment/supply-chain/audits.tomlmaybe. Doesn't matter for the upcoming problem.A while later one of
product-aorproduct-bwant to pull in thelogdependency. They now have the same problem again. They could resolve it way quicker if they could easily see that I already found a trustworthy audit for this crate, and they can rely on my judgement without having to evaluate third parties at all. But there is no tooling for this incargo vetas far as I can see. Maybe I'm responsible for security for all three repos. But I can't remember every crate I ever audited. Do I need to scout around in all our repositories to see if I have already evaluated this dependency or not?Best solution I have come up with so far
What I'd like is a persistent record of all third party audits someone in my organization has decided to approve. And one list of all audits performed within the organization. All projects within the organization can just trust these audits without looking twice. And the latter list (our own audits) can be published on the registry.
Currently the best way for this I have come up with is to set up a
org/supply-chainrepository like this:audits-3rd-party.toml- Manually curated list of approved third party audits, withaggregated-fromfield to show attribution.audits.toml- Manually curated list of all audits performed by all teams at org. This can be published to the public registry for consumption by the entire world. This would be append only (unless we realize we made a mistake).audits-unified.toml- Created bycargo vet aggregate sources.listwheresources.listpoint to the two files above ^. Combining all internal and third party audits that anyone in our org can rely on. This is the file all our code repositories point to in their import statements.~ EDIT: This can be skipped. All the downstream repositories can probably just import both above files. Fewer changes to commit to thesupply-chainrepo that way.And then configure all our products like this (in
product-x/supply-chain/config.toml):