migrate to git submodules

[sjkelly]

The current solution has its merits, but I have had issues with reproducability in the past when I am unaware of patches added to a branch. We could clone specific versions, but at that point we are half-way to a bespoke submodules anyway. Using submodules should enforce version symmetry. It also removes the entries in .gitignore, making whole-project development a bit easier in VSCode. Note this also does shallow clones for CI by default.

This is a quick pass to see what @staticfloat thinks. If this is acceptable, it could be improved by:

• Instantiating in CI only the submodules needed
• Documenting (README and git fetch --unshallow).

[staticfloat]

tl;dr: let's not do this for the kernel or the nvidia driver.

Longer version:

shallow doesn't work. Note how the CI jobs take 10 minutes just to clone down all the submodules: https://buildkite.com/julia-computing-1/xtrx-julia/builds/347#01832381-72ba-473a-8062-31e12c48505f/14-95 (click "show timestamps" to see that it's mostly the kernel causing this slowdown).

I tried to get this working initially as part of #6, you can see my initial attempts, followed by my eventual surrender:

• https://github.com/JuliaComputing/xtrx_julia/commit/2685d16675fcdcac39fe9f6fb35ef90f4c838400
• https://github.com/JuliaComputing/xtrx_julia/commit/321654fe5653bad2974eb3b911821960a65afe5e
• https://github.com/JuliaComputing/xtrx_julia/commit/97d109e340bef54ec0615bbcec2856d6e19966b8

If you can get shallow submodules to work, I'm happy to use them everywhere, but as it stands, burning 10 minutes to clone the kernel every time is a non-starter. I have nothing against using submodules to manage the faster-moving parts such as LMS7002M-driver, and I think that's a good change.

[sjkelly]

Ugh. I see: https://buildkite.com/docs/agent/v3/configuration#no-git-submodules But this seems to be set globally for the runner. So if I understand correctly the problem is not submodules, but rather that build kite clobbers the depth flag?

[staticfloat]

No, see how I used a plugin to skip using buildkite's builtin checkout, then manually called git clone and git submodule update. Even that doesn't work, because, as far as I can tell, the shallow setting just plain does not work. Git always does a full initial clone of the submodule. Have you seen different behavior locally?

[staticfloat]

Or, perhaps this did work (since I committed it) and it was instead that git clone --recursive ignores shallow..... I think perhaps I removed submodules because it was too cumbersome to avoid the initial clone on buildkite...

[staticfloat]

The only thing I'm a little nervous about here is that we don't have a rule to auto-install SoapySDR.jl without the user manually running git submodule update --init; I'll look into the right place to add that.