Option for data variable file structure

[mcexit]

Is your feature request related to a problem? Please describe.

I'd love an configurable option to have the names of my files/folders in my ./.chezmoidata/ be used as key names.

Describe the solution you'd like

Instead of having to do something like this for the .ruff data variable:

./chezmoidata/ruff.jsonc

{
    // <https://docs.astral.sh/ruff/settings/>
    "ruff": {
        // <https://docs.astral.sh/ruff/settings/#ignore>
        "ignore": ["E501"]
    }
}

Instead I could do:

./chezmoidata/ruff.jsonc

// <https://docs.astral.sh/ruff/settings/>
{
    // <https://docs.astral.sh/ruff/settings/#ignore>
    "ignore": ["E501"]
}

Another cool feature would be to have folders be keys as well that merges with a matching filename, so in addition to the above ./chezmoidata/ruff.jsonc file I could add a ./chezmoidata/ruff/ folder with more keys/values:

./chezmoidata/ruff/extend-exclude.jsonc

["tests", "src/bad.py"]

And then get something like this:

$ chezmoi execute-template '{{ .ruff | toPrettyJson }}'
{
  "extend-exclude": [
    "tests",
    "src/bad.py"
  ],
  "ignore": [
    "E501"
  ]
}

I know this is a poor example, but sometimes I end up with large objects that I'd like to separate out and I love the simplicity of using file/folder names to do so. This will also fit well with a directives proposal I plan on making that could exclude certain keys based on conditions.

Describe alternatives you've considered

I've considered whether a plugin or hook could accomplish the same thing.

[halostatue]

This is a good idea, but one that would be backwards incompatible.

A possible path to introduction would be a new configuration variable (chezmoidata.useBasenameAsKey?) that defaults to false (preserving the current behaviour for anyone who does not opt into it), but if set as true it would do the same as you are suggesting.

Consideration could be made for changing the default in Chezmoi 3.0.

A different path forward would be to add a directive (// chezmoi:data-key="ruff") to the top of files you want to opt into this behaviour, but I'm not sure that .chezmoidata/* files parse directives, so that would need specification. This would reduce the amount of nesting in your configuration files, but would not change that you would need to modify the files themselves.

[mcexit]

Thanks... I like the idea of just having a configuration option that defaults to false. If it later became the default then the directives approach makes the most sense to me for users who wouldn't want to opt out entirely, but may have some key names that could potentially conflict with file naming specifications across different operating systems and languages.

[twpayne]

Thank you again for the well-considered proposal!

I considered using filenames as keys when implementing .chezmoidata. Although I agree that using part of the filename as part of the key is superficially neat, there are several complications:

1. This breaks backwards compatibility. You'll need to be backwards-compatible first to avoid breaking existing users, and provide a config option going forward.
2. Fundamentally, this mixes two namespaces: filesystem paths and fields in data structures. The two namespaces have different rules and by conflating them you end up with the combined disadvantages and few of the advantages. For example foo.bar.json is a valid filename, but should this be {"foo.bar":{}} or {"foo":{"bar":{}}}? Equally "foo/bar\0" is a valid JSON property name but not a valid basename, so how would you represent it in the filesystem? As a particularly obtuse example, strings like COM1 are valid filenames on most systems and are valid JSON property names, but are not allowed as a filename on Windows. Filesystems control filenames and there's a whole nightmare here of case (in)sensitivity, case-preserving (or not), drive letters ("C:" or "/mnt/c"?). For a cross-platform dotfile manager like chezmoi, you absolutely cannot trust the filesystem to be even slightly sane, which is why chezmoi only uses simple files and directories in its source state.
3. So, assume that you understand all of the above problems and have a fool-proof method of mapping filesystem paths to fields in a data structure. How are you going to explain all of these rules to chezmoi users? Are you prepared to document this and answer every support request and bug report raised by users who didn't understand the documentation? This is the reality of maintaining popular complex systems.

So, this is why chezmoi has the current approach of merging all data in .chezmoidata at the same level. It's what chezmoi users currently use, it's independent of the vagaries of the filesystem, and it's easy to document and understand. The cost of the current approach is one extra level of data structure in your .chezmoidata files, which, for me, is worth it.

Thanks for reading this long rant and thanks again for your well-thought-through suggestions. I do honestly wish I could say yes to more of them!