Closed polarathene closed 1 year ago
$ git clone --depth=1 https://github.com/docker-mailserver/docker-mailserver .
$ git status
On branch master
Your branch is up to date with 'origin/master'.
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git restore <file>..." to discard changes in working directory)
modified: test/config/ldap/openldap/schemas/postfix-book.ldif
modified: test/config/templates/dovecot-masters.cf
modified: test/config/templates/postfix-accounts.cf
no changes added to commit (use "git add" and/or "git commit -a")
In a git repo, a file can have three representations:
git add
)git commit
)These files were committed prior to the .gitattributes
addition, thus git retained their line endings as-is (CRLF
).
Once .gitattributes
was added, git understood that these files are meant to have LF
inside the repository state - but since the committed version is not LF
, a diff is detected.
CRLF
or LF
in the working tree, this is still detected as a diff (to the repository copy with CRLF
, that git knows needs to be committed as LF
).CRLF
then the now staged (index) copy is converted to use LF
(where the diff is against the repository version with CRLF
).git ls-files --eol
A good way to peek at the EOL usage within the Index and Working Tree by individual files (especially when troubleshooting / verifying .gitattributes
rules) is to run git ls-files --eol
.
We can combine that with grep
to ignore all files recognized as LF
in both the working tree and index:
# It may help to reset both working tree and index state to ensure this represents the repo copy:
$ git reset --hard HEAD
$ git ls-files --eol | grep -vE 'i/lf.*w/lf'
i/-text w/-text attr/-text docs/content/assets/fonts/external-link.woff
i/-text w/-text attr/-text docs/content/assets/logo/.src/dmo-logo-white.png
i/-text w/-text attr/-text docs/content/assets/logo/.src/dmo-logo.png
i/none w/none attr/text docs/content/assets/logo/dmo-logo-white.min.svg
i/none w/none attr/text docs/content/assets/logo/dmo-logo.min.svg
i/-text w/-text attr/-text docs/content/assets/logo/favicon-32x32.png
i/-text w/-text attr/-text docs/overrides/assets/img/bg-water.webp
i/-text w/-text attr/-text docs/overrides/favicon.ico
i/none w/none attr/text target/postfix/header_checks.pcre
i/ w/ attr/text=auto test/bats
i/crlf w/crlf attr/text test/config/ldap/openldap/schemas/postfix-book.ldif
i/crlf w/crlf attr/text test/config/templates/dovecot-masters.cf
i/mixed w/mixed attr/text test/config/templates/postfix-accounts.cf
i/none w/none attr/text test/config/without-accounts/.gitkeep
i/ w/ attr/text=auto test/test_helper/bats-assert
i/ w/ attr/text=auto test/test_helper/bats-support
-text
is binary content, so that's all expected.none
means the file has a single line (potentially an empty file), EOL is missing (would add a blank line at end of file).mixed
is usually not good. There is both CRLF
and LF
in that file, but thankfully with .gitattributes
it was caught and being fixed (there are some files where mixed
is valid, but it's rare).NOTE: As I used git reset
in the example above, both columns (i/
index + w/
working tree) are the same. However, on Windows when I checkout a branch:
i/
matches the repo committed EOL (normalized now by .gitattributes
rules)w/
checks-out text files with CRLF
(when .gitattributes
doesn't assign a specific eol
, global git config defaults core.autocrlf=false
+ core.eol=native
are used as fallback, hence CRLF
on Windows), but only files that have a diff between the two branches/commits.
$ git ls-files --eol | grep -vE 'i/lf.*w/lf' | wc -l
247
# Compared to total files tracked by git which are checked out with LF:
$ git ls-files | wc -l
422
The amount depends on how many files changed between the branches.
EDIT: Not exactly, seems to be a specific kind of diff š¤·āāļø
git reset
would correct that (although I ran that via WSL, while the checkout was via a Windows GUI client).
Prior to adding .gitattributes
this resulted in some files that were LF
being opened with CRLF
in VSCode. I wasn't sure at the time what contributed to that.
.gitattributes
this is a non-issue as all text files can use LF
(text
attribute) when in the index and repo, and when necessary enforce the working-tree EOL with eol=lf
/ eol=crlf
..gitattributes
this way, makes the EOL in the working tree less of a concern, so the fallback core.eol=native
can accommodate any local developer software that expects only LF
or CRLF
on that platform.eol
enforced in .gitattributes
for consistency, or be generated š¤·āāļø core.eol=native
does not make sense, nor core.eol=lf
/ core.eol=crlf
; One can instead prefer core.autocrlf=input
which will not perform any output conversion (overrides core.eol
setting), so the working-tree checkout copy should match the committed EOL.git config --global core.autocrlf input
will do this, but AFAIK changes the default for all git projects to normalize text EOL (crlf
/ mixed
) to lf
when staging changes to commit (which is probably what you'd want most of the time)..editorconfig
with an editor that supports it (such as VSCode with the official extension installed). This only applies the EOL conversion upon save however, thus not foolproof of working-tree gotchas.git add --renormalize .
git add --renormalize .
will automate resolving this.
Although I think staging the changes with just git add .
may work in most cases? š¤·āāļø (I wanted to be sure, so discovered --renormalize
from this SO answer, while this SO answer further explains it normalizes across all tracked files)
Running git rm -r --cached . && git reset --hard HEAD
is needed afterwards (warning: Destructive to (un)staged changes).
CRLF
conversion to LF
by adding to the index, (instead of making the change prior via explicit conversion in the working tree) would still have the CRLF
present in files) to reflect the EOL of now updated repository copy.
Description
The
contributors.yml
scheduled workflow recently failed which caught a mishap (my fault) withCRLF
line endings committed in these files:test/config/ldap/openldap/schemas/postfix-book.ldif
test/config/templates/dovecot-masters.cf
test/config/templates/postfix-accounts.cf
Insights:
.gitattributes
was added to prevent files checking inCRLF
line endings.linting.yml
workflow did not catch this mistake either (even though config implies it should have).make lint
actually uses thisecrc.json
config instead which excludes linting thetest/
directory.test/
folder was excluded by: https://github.com/docker-mailserver/docker-mailserver/pull/2000Logs
Contributors failing workflow log
https://github.com/docker-mailserver/docker-mailserver/actions/runs/6287887262/job/17072802215#step:6:53 ``` Create or update the pull request branch /usr/bin/git symbolic-ref HEAD --short master Working base is branch 'master' /usr/bin/git checkout --progress -B 7a7933b5-ccd5-4ecd-8447-70a6b72ee885 HEAD -- Switched to a new branch '7a7933b5-ccd5-4ecd-8447-70a6b72ee885' M test/config/ldap/openldap/schemas/postfix-book.ldif M test/config/templates/dovecot-masters.cf M test/config/templates/postfix-accounts.cf /usr/bin/git status --porcelain -unormal -- M test/config/ldap/openldap/schemas/postfix-book.ldif M test/config/templates/dovecot-masters.cf M test/config/templates/postfix-accounts.cf Uncommitted changes found. Adding a commit. /usr/bin/git add -A warning: in the working copy of 'test/config/ldap/openldap/schemas/postfix-book.ldif', CRLF will be replaced by LF the next time Git touches it warning: in the working copy of 'test/config/templates/dovecot-masters.cf', CRLF will be replaced by LF the next time Git touches it warning: in the working copy of 'test/config/templates/postfix-accounts.cf', CRLF will be replaced by LF the next time Git touches it /usr/bin/git -c author.name=github-actions[bot] -c author.email=41898282+github-actions[bot]@users.noreply.github.com -c committer.name=github-actions[bot] -c committer.email=41898282+github-actions[bot]@users.noreply.github.com commit -m docs: update `CONTRIBUTORS.md` --signoff [7a7933b5-ccd5-4ecd-8447-70a6b72ee885 3[57](https://github.com/docker-mailserver/docker-mailserver/actions/runs/6287887262/job/17072802215#step:6:63)8b7b] docs: update `CONTRIBUTORS.md` 3 files changed, 18 insertions(+), 18 deletions(-) ``` https://github.com/docker-mailserver/docker-mailserver/actions/runs/6287887262/job/17072802215#step:6:487 ``` From https://github.com/docker-mailserver/docker-mailserver * [new branch] contributors-update -> origin/contributors-update Pull request branch 'contributors-update' already exists as remote branch 'origin/contributors-update' /usr/bin/git checkout --progress contributors-update -- Switched to branch 'contributors-update' M test/config/ldap/openldap/schemas/postfix-book.ldif M test/config/templates/dovecot-masters.cf M test/config/templates/postfix-accounts.cf /usr/bin/git rev-list --right-only --count master...7a7933b5-ccd5-4ecd-8447-70a6b72ee885 1 /usr/bin/git rev-list --right-only --count master...contributors-update 0 /usr/bin/git diff --quiet contributors-update..7a7933b5-ccd5-4ecd-8447-70a6b72ee885 Resetting 'contributors-update' /usr/bin/git checkout --progress -B contributors-update 7a7933b5-ccd5-4ecd-8447-70a6b72ee885 -- error: Your local changes to the following files would be overwritten by checkout: test/config/ldap/openldap/schemas/postfix-book.ldif test/config/templates/dovecot-masters.cf test/config/templates/postfix-accounts.cf Please commit your changes or stash them before you switch branches. Aborting Error: The process '/usr/bin/git' failed with exit code 1 ```Example of ECLint successfully catching this rule elsewhere
- Caught by a PR that introduced other files with CRLF by me in the `target/` folder: https://github.com/docker-mailserver/docker-mailserver/pull/3524#issuecomment-1704744596 - `.gitattributes` PR comment referencing it as justification for why the config was worth making explicit in git: https://github.com/docker-mailserver/docker-mailserver/pull/3527#issuecomment-1712225729 Workflow log: https://github.com/docker-mailserver/docker-mailserver/actions/runs/6070286374/job/16466057026#step:5:24 ``` Status: Downloaded newer image for mstruebing/editorconfig-checker:2.7.0 target/features/ldap/dovecot.base: Wrong line endings or no final newline Not all lines have the correct end of line character target/features/ldap/dovecot.tmpl: Wrong line endings or no final newline Not all lines have the correct end of line character target/features/ldap/postfix.base: Wrong line endings or no final newline Not all lines have the correct end of line character target/features/ldap/postfix.tmpl: Wrong line endings or no final newline Not all lines have the correct end of line character target/features/ldap/saslauthd.base: Wrong line endings or no final newline Not all lines have the correct end of line character target/features/ldap/saslauthd.tmpl: Wrong line endings or no final newline Not all lines have the correct end of line character 12 errors found [ ERROR ] ECLint failed make: *** [Makefile:76: eclint] Error 1 ```