Commiting issues with case sensitivity in filenames

[MichalVasut]

Hello, there seems to be issue with case sensitivity of file names during git add...

$ ls
bumpver.toml  VERSION

$ cat bumpver.toml 
[bumpver]
current_version = "1.2.10"
version_pattern = "MAJOR.MINOR.PATCH"
commit_message = "bump version {old_version} -> {new_version}"
commit = true
tag = false
push = false

[bumpver.file_patterns]
"VERSION" = [
    '{version}',
]

$ cat VERSION 
1.2.10

$ bumpver update --patch --no-tag-commit -vv
2023-05-03T09:17:59.177 DEBUG   bumpver.cli       - Logging configured.
2023-05-03T09:17:59.261 DEBUG   bumpver.config    - Config Parsed: Config(
    current_version='1.2.10',
    version_pattern='MAJOR.MINOR.PATCH',
    pep440_version='1.2.10',
    commit_message='bump version {old_version} -> {new_version}',
    commit=True,
    tag=False,
    push=False,
    is_new_pattern=True,
    file_patterns={
        'bumpver.toml': 'current_version = "MAJOR.MINOR.PATCH"',
        'version': 'MAJOR.MINOR.PATCH',
    }
)
2023-05-03T09:17:59.291 DEBUG   bumpver.vcs       - vcs found: git
2023-05-03T09:17:59.291 INFO    bumpver.vcs       - fetching tags from remote (to turn off use: -n / --no-fetch)
2023-05-03T09:17:59.291 DEBUG   bumpver.vcs       - git branch -vv
2023-05-03T09:17:59.326 DEBUG   bumpver.vcs       - git config --get remote.origin.url
2023-05-03T09:17:59.358 DEBUG   bumpver.vcs       - git tag --list
2023-05-03T09:17:59.393 DEBUG   bumpver.vcs       - ls_tags output ['1.0.1', '1.0.2', '1.1.0', '1.2.0']
2023-05-03T09:17:59.394 DEBUG   bumpver.cli       - found tags: 1.2.0, 1.1.0, 1.0.2 ... (4 in total)
2023-05-03T09:17:59.394 INFO    bumpver.cli       - Using pattern MAJOR.MINOR.PATCH
2023-05-03T09:17:59.395 INFO    bumpver.cli       - regex = re.compile(r"""
    (?P<major>[0-9]+)
    \.
    (?P<minor>[0-9]+)
    \.
    (?P<patch>[0-9]+)
""", flags=re.VERBOSE)
2023-05-03T09:17:59.398 INFO    bumpver.cli       - Old Version: 1.2.10
2023-05-03T09:17:59.399 INFO    bumpver.cli       - New Version: 1.2.11
--- bumpver.toml
+++ bumpver.toml
@@ -1,5 +1,5 @@
 [bumpver]
-current_version = "1.2.10"
+current_version = "1.2.11"
 version_pattern = "MAJOR.MINOR.PATCH"
 commit_message = "bump version {old_version} -> {new_version}"
 commit = true
--- version
+++ version
@@ -1 +1 @@
-1.2.10
+1.2.11
2023-05-03T09:17:59.441 DEBUG   bumpver.vcs       - git status --porcelain
2023-05-03T09:17:59.478 DEBUG   bumpver.vcs       - git add --update 'version'
2023-05-03T09:17:59.512 DEBUG   bumpver.vcs       - git add --update 'bumpver.toml'
2023-05-03T09:17:59.553 INFO    bumpver.vcs       - git commit --message 'bump version 1.2.10 -> 1.2.11'

$ git status
On branch master
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   VERSION

no changes added to commit (use "git add" and/or "git commit -a")

When I remae the file to lower case (version), it works as expected.

[mbarkhau]

Thanks for the report, that does indeed look like an issue.

[mbarkhau]

Unfortunately I've not been able to reproduce this. Specifically, on my system there seems to be a difference in how the configuration is parsed:

2023-05-04T15:54:12.583 DEBUG   bumpver.config    - Config Parsed: Config(
    current_version='1.2.10',
    version_pattern='MAJOR.MINOR.PATCH',
    pep440_version='1.2.10',
    commit_message='bump version {old_version} -> {new_version}',
    commit=True,
    tag=False,
    push=False,
    is_new_pattern=True,
    file_patterns={
        'VERSION': 'MAJOR.MINOR.PATCH',
        'bumpver.toml': 'current_version = "MAJOR.MINOR.PATCH"',
    }
)

As compared to yours:

    file_patterns={
        'bumpver.toml': 'current_version = "MAJOR.MINOR.PATCH"',
        'version': 'MAJOR.MINOR.PATCH',
    }

Can you update to 2023.1121 and try again, or somehow narrow down the issue?

The tests also run on MacOS in the CI system, so if you can write a test that triggers the issue in the CI, then that would be helpful.

[MichalVasut]

Can you update to 2023.1121

Unfortunately I cannot, currently I don't have personal device with Windows, it's from work, where we have restricted rights to make system update. (if we are talking about Windows version and I'm not even sure what version is installed - I'll check it later).

or somehow narrow down the issue?

I'll try to pinpoint it from soitcecode, it should be imho something with that name parsing or handling, that is different across platforms...

[jscooksey]

I've just noticed this on my Windows based system. That it didn't commit files, and when I look back it is all files with upper and lower case names that didn't get added in for the bumpver commit.

Version in use:

$ bumpver --version
bumpver, version 2023.1121

Commit done by bumpver:

$ git show --pretty="" --name-only 61b4d9ae6e13db132414cffedf0408581eff97e9
bumpver.toml
netbox/minder/__init__.py
netbox/minder/choices.py
netbox/minder/interogatecore.py
netbox/minder/minder.py
netbox/minder/radiuscreds.py
netbox/reports/__init__.py
netbox/scripts/__init__.py
netbox/scripts/add_fibre.ios.j2
netbox/scripts/add_nbn.ios.j2
netbox/scripts/add_sip.j2
netbox/scripts/hosting.ios.j2
netbox/scripts/mindertenants.py
netbox/scripts/template.ios.j2

Following commit, with no actual changes made to these files since the bumpver commit:

$ git show --pretty="" --name-only a7004bd1295108b269c99dc646b3af5eb509a51a
netbox/reports/DeviceRacking.py
netbox/scripts/AddDeviceTypeComponents.py
netbox/scripts/MinderTools.py
netbox/scripts/ServiceFibre.py
netbox/scripts/ServiceHosting.py
netbox/scripts/ServiceNBN.py
netbox/scripts/ServiceSIP.py

[mbarkhau]

Thanks for the extra info @jscooksey.

If you have any ability to debug this further, I'd greatly appreciate it.

[jscooksey]

Yes, I'll take a bit more of a look at it this evening. I also have a MacOS machine as well, so I'll see if that's doing the same thing.

[jscooksey]

Just quickly I pulled the /test/fixtures/project_b folders

• Added Hello.py and lower.py to each subfolder including the version string
• Modified to the setup.cfg file so it would update them as well
• Git init, and pushed up to project_b on GitHub
• Then did bumpver update

Same as before, it added all lower case including the new lower.py, but didn't add any that had upper case, including README.rst byt the looks

Anyway, I'll have more time to look through it later.

[jscooksey]

On MacOS it hasn't had the same issue, and has worked fine Python is 3.10.6 whereas the Windows 11 machine has 3.9.x Both have the same version of bumpver

git show --pretty="" 2391d2f81d909b74930539d39e79390e7ac4d6e9 --name-only
bumpver.toml
netbox/minder/__init__.py
netbox/minder/choices.py
netbox/minder/interogatecore.py
netbox/minder/minder.py
netbox/minder/radiuscreds.py
netbox/reports/DeviceRacking.py
netbox/reports/__init__.py
netbox/scripts/AddDeviceTypeComponents.py
netbox/scripts/MinderTools.py
netbox/scripts/ServiceFibre.py
netbox/scripts/ServiceHosting.py
netbox/scripts/ServiceNBN.py
netbox/scripts/ServiceSIP.py
netbox/scripts/__init__.py
netbox/scripts/add_fibre.ios.j2
netbox/scripts/add_nbn.ios.j2
netbox/scripts/add_sip.j2
netbox/scripts/hosting.ios.j2
netbox/scripts/mindertenants.py
netbox/scripts/template.ios.j2

[jscooksey]

I tried setting git global options on the Win11 machine git config --global core.ignorecase true just in case it was to do with that. But true or false made no difference and bumpver didn't add the files with upper case in the names.

I tried to get a fork working on the Win11 system as well, but had a few issues with that and Conda and make. In just doing pytest I got folder access denied for temp folders, which seemed a bit odd, as the folders are under my profile so should have been fine. But I didn't dig to much further into it tonight.
Walked in the code a little, vcs.py#LINE252 I'll try and have more of a crack later

[jscooksey]

OK, it seems to be the call out to git to add files under Windows. From the looks of some of the output as it runs, it appears filenames are all taken to lowercase during processing within bumpver. So when it call out git to add the file UpperCase.py its calling: git add --update uppercase.py

Current git config is core.ignorecase=false

The Windows version of git must looks for matching case, as it errors out not finding the file as shown manually below:

$ git commit
[main ebc1774] yep
 1 file changed, 4 insertions(+)
$ git diff --name-only --cached
$ nano netbox/wip/UpperCase.py
#
# Changed the content 
#
$ git add --update netbox/wip/uppercase.py
fatal: pathspec 'netbox/wip/uppercase.py' did not match any files
$ git diff --name-only --cached
$ git add --update netbox/wip/UpperCase.py
warning: in the working copy of 'netbox/wip/UpperCase.py', LF will be replaced by CRLF the next time Git touches it
$ git diff --name-only --cached
netbox/wip/UpperCase.py
$ git commit
[main 3291f08] again
 1 file changed, 1 insertion(+), 1 deletion(-)
$

With git config is core.ignorecase=true it didnt throw the did not match any files error

$  git diff --name-only --cached
$ git add --update netbox/wip/uppercase.py
$  git diff --name-only --cached
$ git add --update netbox/wip/UpperCase.py
warning: in the working copy of 'netbox/wip/UpperCase.py', LF will be replaced by CRLF the next time Git touches it
$  git diff --name-only --cached
netbox/wip/UpperCase.py

[yqbear]

I've been seeing this same issue on my windows machine. I see that all my filenames in bumpver.file_patterns are being returned in lowercase. For my machine, I've tracked this to the use of pathlib2. In _iter_glob_expanded_file_patterns line 255 in config.py is the line:

filepaths = list(pl.Path().glob(filepath_glob))

If I have a filepath_glob that has mixed case like "MixedCaseFile.txt", then this line returns a WindowsPath with the name in all lowercase. I can look into this more, but this seems to be the cause of the error for me.

I'm using a test in test_cli.py to check this right now.

def test_filename_case(request, monkeypatch):
    runner = CliRunner()
    monkeypatch.chdir(request.path.parent / "fixtures" / "project_e")
    result = runner.invoke(cli.cli, ["update", "--dry", "-n", "--patch"])
    assert result.exit_code == 0
    assert "MixedCaseFile.txt" in result.output.rstrip()

The project_e fixture has a file named MixedCaseFile.txt and a bumpver.toml file with the following contents.

[bumpver]
current_version = "0.0.0"
version_pattern = "MAJOR.MINOR.PATCH"
commit_message = "bump version {old_version} -> {new_version}"
commit = true
tag = false
push = false 

[bumpver.file_patterns]
"bumpver.toml" = [
    'current_version = "{version}"',
]
"MixedCaseFile.txt" = [
    '{version}'
]

[yqbear]

One workaround for now is to add an * after the filename to make it a glob pattern. pathlib2 doesn't do case folding for this. Caution should be taken that you won't pick up other files. I've tried this with the current version and it worked.

[bumpver.file_patterns]
"bumpver.toml" = [
    'current_version = "{version}"',
]
"MixedCaseFile.txt*" = [
    '{version}'
]
"VERSION*" = [
    '{version}'
]

[SRFU-NN]

I have raised the issue with pathlib2: https://github.com/jazzband/pathlib2/issues/86