Support multi-repo application scenarios

[savannahostrowski]

Applicable to:

• Microservice architectures (e.g. how do I deploy an application where not all of the code lives in a single repo)
• Kubernetes apps (per the AKS team, a common scenario is that a team might have their manifests in one repo, IaC in another repo, and app code in another repo).

Off the top of my head, a way we could approach this is by allowing GH URLs as paths for repos which contain app code, IaC code, manifests, etc. We could also permit this at the service level. This would enable a team to have a single repo that really just contains an azure.yaml and then azd does it's magic to pull everything in.

[wbreza]

Another idea could be a new azd project repo that uses git submodules to pull in the repos for the related IaC & code.

[savannahostrowski]

I have personally never had a good /straightforward development experience with git submodules. I think I was also chatting with @pamelafox about this fairly recently for an AI template we were chatting about. Would love to hear her opinion as well.

[pamelafox]

When I was at Khan Academy, we used a single private repo for our main (Python) code, and then we used a git submodule to bring in other things, like the coding environment. For whatever reason, we constantly had git conflicts over the submodules. I would not particularly want to go down that route again.

That being said, if a company already uses submodules for multi-repo applications and is fine with that, then I would assume azd would still work for them - i.e. no changes should be needed to azd to make it work with submodules, right?

[jwhitley]

Chiming in as a new azd user who is working through transitioning a company from an early ad-hoc Azure deployment to one that's properly reproducible. Our code fairly naturally falls into multiple separate (and already existing) repositories that represent a small number of services residing on Azure. FWIW, I have had a lot of experience with varied version control systems over the years, and I'll agree about issues with git submodules: they are very much not designed for collaboration.

The problem is that the .gitmodules file in the parent repository contains gitlink lines which have the git SHA the target repository is expected to be at. In a normal collaborative environment where devs expect to work on submodules through the context of the parent module, this means a mandatory and constant stream of conflicts over the git SHA values for each submodule. The basic scenario of:

• Develeoper One works on a feature in parent/submodule/A
• Developer Two also works on a feature on the same submodule

Is now a guaranteed conflict: one of them lands the feature first, then the other has to resolve the conflict on the SHA for submodule A to get their feature in. It just gets worse the more submodules and collaboration across them there is.

There are patterns that work. An old tool called giternal (my fork, here) was similar to submodules, but took a spec which contained branch names rather than git SHAs. I helped build a devops setup for a client with a very complex backend (dozens and dozens of services) that used giternal on the central devops repo to enable collaboration. The model worked like this:

• A developer/sub-team would create feature branch(es) for the service(s) their feature touched.
• A matching feature branch would be created on the parent devops repo, and the giternal YAML spec would be updated to point to the feature branches created in the prior step.
• The dev(s) would run giternal update to clone/update all of the dependencies in the giternal spec into the parent repository (pretty much like git submodule does).

This had a tidy workflow: the devops tooling we built allowed "one big button" deployment of the entire shebang into a local VM, or onto a single cloud VM instance (great for end-of-sprint team demos). The backing devops specs (similar to say Azure ARM templates or Bicep files, but older tech) were directly used by the operations team for development cluster or full production deployments – this was the IaC path of direct communication between the developers updating deployment code and the operations team (who vetted all IaC changes on the central repo).

A key thing w.r.t. the submodule problems above: those feature branches on the parent/central devops repo were throwaway. This eliminated the possibility of a conflict. Those feature branches were only needed to develop and vet out "the constellation of services with ONLY this feature's changes". Essentially, the giternal spec on the main branch ALWAYS pointed to the main branches of the child repos. When a feature was vetted to land, all of that feature's branches across the service repos were merged into their respective main branches at the same time. (and questions like "what did we deploy/release at time X" were largely delegated to CI and ops tooling, if I recall correctly. CI would record "this is a tested and green-lit set of services (git SHAs)", ops punches The Big Button on the desired release set to roll forward/back.

Anyhow, I hope some of that helps to illuminate the problem space for this feature, and perhaps sparks some good ideas!

[kristenwomack]

Thank you for sharing your experience @jwhitley - super helpful as we are looking at this scenario.