`chezmoi unmanaged` returns subfiles/subdirs of .chezmoiexternal-managed directories

[XaF]

Describe the bug

If a directory is managed through .chezmoiexternal, the directory itself appears when calling chezmoi managed, but its subfiles and subdirectories appear in chezmoi unmanaged. If those files/directories are not part of the "import" made by chezmoi, I would understand the output. But other than that (either part of imported archive or git-repo) these should be considered managed too.

To reproduce

• Add an externally-managed git repo
• Run chezmoi managed, the path to the managed directory is there
• Run chezmoi unmanaged, the paths to everything under the managed directory are there

Expected behavior

• chezmoi unmanaged should not return any of the underlying files and directories, so long as:
  • they were imported by chezmoi (e.g. for git: they are tracked by git, and are not the .git directory)
  • they haven't been modified since last import by chezmoi (e.g. for git: they are not in git diff/status) - actually not sure if this one would flag as unmanaged, since the file is still managed but just not in sync right now

This could be handled by chezmoi caching hashes of all the files imported through .chezmoiexternal in its cache, allowing for comparisons. Ideally, we would have comparison process that are type-specific (e.g. archive and type would use the hash approach, git-repo could simply check through git commands and status, etc.)

Additional context

Put this as a bug report (since reporting unmanaged files that are, actually, managed), but maybe this is a feature request?

[twpayne]

This is the same as #2778. I agree that the current behavior seems counter-intuitive, but it is actually correct. See https://github.com/twpayne/chezmoi/issues/2778#issuecomment-1438185488.

tl;dr don't use git-repo externals unless you really, really have to.

[XaF]

From your other thread (not sure if I should write there or here):

This means that chezmoi is limited to running git init and git pull in the directory, and chezmoi has no insight into the contents of the directory.

Why that limitation ? Why not using git ls-files and git diff or git status ?

[twpayne]

Why that limitation ? Why not using git ls-files and git diff or git status ?

Because to implement this would require chezmoi to duplicate a lot of git's logic, including all of git's edge cases, and git is sufficiently complex that I don't want to go down this path.

For example, how would you handle files in the external git repo working copy but are in .gitignore? Are they managed by git or not? Are they managed by chezmoi?

Often, .gitignore is used for files that are generated when you run the software (e.g. *.pyc if you run a Python script) but are should not be checked into the git repo. Are *.pyc files managed by git, chezmoi, both, or neither?

How about files which are in .gitignore and also in the git repo?

How do ensure that this works correctly with git submodules, shallow clones, git-lfs, and chezmoi's builtin git?

If you could come up with a PR that implements all of the above correctly, then I would consider it.

Edit: grammar

[XaF]

What about considering that everything under the path of an external git-repo is just considered 'managed' ? I see more of a usecase where chezmoi unmanaged tells me what new files I have in my home directory that I might have forgotten to manage, and in the current situation chezmoi unmanaged is full of git-repo subfiles.

Then the limit of using git-repo is that you don't have anything under that directory which is managed by chezmoi, but I would consider this is a better limitation than the opposite (seeing all the files as unmanaged). And of course, we could document that aspect, with the justification that in that directory, we do let git manage the files, as the person has asked for that directory to be a git repo!

[twpayne]

What about considering that everything under the path of an external git-repo is just considered 'managed' ?

OK, this makes sense. I agree that it's a better user experience than the current behavior.