Best practices for git repository containing multiple modules

[bmaximuml]

I've got a single repository containing multiple modules at different distances down the tree (structure below). I'm trying to work out the best values to supply to GIT_PROVIDER_CONFIG to make it as easy as possible to set up these modules. So far I have something like what's below, but terrareg won't accept it because it wants both {namespace} and {module} in the clone_url.

Have I got the right idea here, or am I missing something obvious?

{"name": "multiple modules", "base_url": "https://github.com/example/{namespace}/tree/main/dir1/{module}/terraform", "clone_url": "ssh://git@github.com/vivacitylabs/{namespace}.git", "browse_url": "https://github.com/vivacitylabs/{namespace}/tree/{tag}/cloud/{module}/{path}", "tag_format": "v{major}.{minor}"}

Repo layout:

/
==> /dir1
=======> /dir1/nest1/module1
=======> /dir1/nest1/module2
==> /dir2
=======> /dir2/module3

[MatthewJohn]

Hey @bmaximuml,

From how Terrareg was designed to handle these configurations, in this scenario, the configuration would ideally be:

{
    "name": "multiple modules",
    "base_url": "https://github.com/example/{namespace}",
    "clone_url": "ssh://git@github.com/vivacitylabs/{namespace}.git",
    "browse_url": "https://github.com/vivacitylabs/{namespace}/tree/{tag}/{path}",  // I'm not sure what the 'cloud' part here was - I assume was meant to be dir1?
    "tag_format": "v{major}.{minor}"
}

And after this, each module is configured with the "git_path" of dir1/{module_name}.

I have a couple of thoughts on this:

• The validation of the the git provider will not allow these URLs without module/provider, as you stated
• The git path has to be provided in each of the modules

For me, I think the validation is useful (to a degree), as it stops accidental misconfiguration (though I guess we allow this similarly with the tag_format, where no longer using {version} can lead to reduced functionality, but we do allow it). If it were configured on an API, I'd be tempted to warn users with some kind of configuration - but since it's configuration, maybe printing a warning, but not sure if it would be seen.

The other consideration is whether the have a "git_path_template" in the git provider config, which in your case would become:

  ...
  "git_path": "dir1/{{module}}"
  ...

Would also have to consider in this case whether, if the git provider containers a git_path template, whether it stops the user from specifying one in the module configuration.

Matt

[MatthewJohn]

Created gitlab issue: https://gitlab.dockstudios.co.uk/pub/terrareg/-/issues/502 gitlab-issue-id:502

[bmaximuml]

I agree that the validation is useful, and should be kept. I reckon the introduction of the git_path_template could expand the validation so either the existing validation needs to pass, or the validation should apply to effectively the value of git_path_template appended to the url, so that e.g. if {module} is not provided in the URL, it must be provided in the git_path_template?

[MatthewJohn]

@bmaximuml That sounds good - adding a git_path_template would be good - though if it still validates {provider}, it may still not fit your use-case? (Though I'll check if it currently validates this)

[bmaximuml]

though if it still validates {provider}, it may still not fit your use-case?

What do you mean? I don't see {provider} in the above conf

[MatthewJohn]

What do you mean? I don't see {provider} in the above conf

Ah, apologies, I had thought that is was a required placeholder of the URLs, but it's already optional :)

I will take a look at getting this working

Matt

[MatthewJohn]

Hey @bmaximuml - this has now been released in v3.4.0 :)

I'll close for now - let me know if you have any issues/questions though!

Matt