Use git submodules as a source of dependencies

[mmjconolly]

what

• If repository lacks wit-manifest.json, use git submodules to collect dependency data
• wit-manifest.json is used in preference to git submodules, for repositories that have both
• wit add-dep/wit update-dep on a repository that only has git submodules errors out with a message, user would use regular git submodules commands to update those dependencies. These are the only two commands that should be impacted.

why

• in repositories that already keep a wit-manifest.json and a list of submodules manually sync-ed, we could delete the wit-manifest.json file (presuming they used a new enough wit version). It might not make sense in all cases to do this but the option is there

how

When wit is performing a resolve to determine which is the correct commit to checkout, it checks for the wit-manifest as usual, if this is missing it checks for a .gitmodules file. The .gitmodules file provides the list of remote url and the within-repository path, for each submodule. The within-repository path is used to ask the git index what the pointed-to commit is for the submodule.

relationship to submodules

This could be viewed as creating a read-only on-demand wit-manifest.json from the submodule information. Wit makes no attempt to write submodule information. Submodules do appear in the wit-lock.json, but wit does not know they are submodules.

[mmjconolly]

do you know if this will cause us to add additional Git repos to the wit workspace for any of our larger, internal repositories? I'm just wondering if there are any repositories which do not have a wit-manifest.json file but have Git submodules that we may not want to pull into the workspace.

In the current scenario, where all the repositories that we care about have wit-manifest.json, I've noticed only a couple of added documentation repositories

I'd like to avoid cases like in rocket-chip where the full Git submodule dependency tree pulls in the entirety of the full GCC Git repository, which is a pretty heavy thing to pull in. rocket-chip specifically will not have an issue because it has a wit-manifest.json file, but I want to make sure we don't cause any other large repos like that to get pull in like this.

I guess we can look at that if (when) we decide to delete the wit-manifest on rocket-chip. (I just did a recursive submodule update on rocket-chip, couldn't see a gcc?)

[richardxia]

I guess we can look at that if (when) we decide to delete the wit-manifest on rocket-chip. (I just did a recursive submodule update on rocket-chip, couldn't see a gcc?)

We must have removed that at some point, probably after GCC upstream finally accepted the RISC-V patches. That at least simplifies this problem a lot!