[feature request] add ability to include other TOML files containing subsets of configuration.

[romainkomorndatadog]

Feature description

The ability to separate hatch.toml into separate, distinct files would be a useful way to organize separate environments.

Why?

For mono-repos with large number of individual components, hatch.toml can get prohibitively long.

One example is dd-trace-py's growing use of Hatch to replace riot. Our current riotfile.py is almost 3000 lines long (due to a wide variety of test environments / matrices). Migrating all of that into our existing hatch.toml would likely push that file past the thousand line mark (depending how well we make use of matrices and overrides).

Possible implementations

hatch.d directory

My naive example of how this could look from a user's perspective would be to have a hatch.d directory containing more .toml files, for example:

repo/
  hatch.toml
  hatch.d/
    lint.toml
    srvc1_envs.toml
    srvc2_envs.toml
...

The idea would be that:

• any .toml file in hatch.d would be independent from all other files under hatch.d
• the overall resulting configuration would effectively be a concatenation hatch.toml followed by all the .toml files in hatch.d

An extra functionality to be able to have recursive directories might be desirable (since trading a long hatch.toml file for one directory with a long of individual files arguably is more of the same problem).

include statements

Another possibility would be to add a configuration directive to include other files, eg:

[envs.my_included_env]
include_toml = "path/to/my_included_env.toml"

Although in cases where there are large numbers of environments, that could still prove to be a lot and make the hatch.toml file long.

[romainkomorndatadog]

The idea to limit this to environments is more of a "for the sake of simplicity" thing (and because that's my current use case). I imagine other people/orgs might have different ways of thinking about how they could split things up.