This PR fixes a pretty nasty bug on Windows where a file would be added to the annex and then also to git, causing the pointer (link) file in annex to break. The content would still remain in the repository, but the linking was wrong.
The cause of the problem was due to the way file paths are filtered during a commit. When adding files (either new or existing ones) to the index, the client first calls git annex add with the filtering parameters. This adds all files that don't match the git-annex filtering rules to the annex. After that is done, git add is run on the rest of the files. On Linux we can git add everything and it wont make a difference (annexed files wont get added again), but on Windows, the file in the working tree is unlocked (Windows works in direct mode) so a git add adds the whole binary file and overwrites the existing link file that was added by git-annex. So to avoid this, the client would get a list of all annexed files (including newly added ones) and filter the path arguments so that git add is run only on files that are not annexed. The bug was that the filtering function did not take into account path formatting differences between Windows and Linux. Git and git-annex both return Unix paths (/ separator), so on Windows, if a path contained a \, it wouldn't match the known paths from git-annex and the original problem persisted.
Path collection and filtering now runs filepath.Clean() on all path strings to unify their formatting.
With this PR we also begin running tests on Appveyor for Windows. Currently, only a new test is run which was written to catch the bug described above. I will soon be adapting tests so they can run on Windows without issue, or I will configure Appveyor to run a containerised GIN (GOGS) server as we do on Travis.
A smaller change is that I replaced the semver parser we were previously using (https://github.com/hashicorp/go-version) with a very lax, self written function. On many platforms, git and git-annex both have version strings which don't follow semver rules (e.g., git version 2.20.1.windows.1 or 6.20171205-ge5c610b29PS), so it doesn't make sense to try to work around and use an external dependency. This should fix all previous issues with version parsing (closes #121 and #147).
This PR fixes a pretty nasty bug on Windows where a file would be added to the annex and then also to git, causing the pointer (link) file in annex to break. The content would still remain in the repository, but the linking was wrong.
The cause of the problem was due to the way file paths are filtered during a commit. When adding files (either new or existing ones) to the index, the client first calls
git annex addwith the filtering parameters. This adds all files that don't match the git-annex filtering rules to the annex. After that is done,git addis run on the rest of the files. On Linux we cangit addeverything and it wont make a difference (annexed files wont get added again), but on Windows, the file in the working tree is unlocked (Windows works in direct mode) so agit addadds the whole binary file and overwrites the existing link file that was added by git-annex. So to avoid this, the client would get a list of all annexed files (including newly added ones) and filter the path arguments so thatgit addis run only on files that are not annexed. The bug was that the filtering function did not take into account path formatting differences between Windows and Linux. Git and git-annex both return Unix paths (/separator), so on Windows, if a path contained a\, it wouldn't match the known paths from git-annex and the original problem persisted.Path collection and filtering now runs
filepath.Clean()on all path strings to unify their formatting.With this PR we also begin running tests on Appveyor for Windows. Currently, only a new test is run which was written to catch the bug described above. I will soon be adapting tests so they can run on Windows without issue, or I will configure Appveyor to run a containerised GIN (GOGS) server as we do on Travis.
A smaller change is that I replaced the semver parser we were previously using (https://github.com/hashicorp/go-version) with a very lax, self written function. On many platforms, git and git-annex both have version strings which don't follow semver rules (e.g.,
git version 2.20.1.windows.1or6.20171205-ge5c610b29PS), so it doesn't make sense to try to work around and use an external dependency. This should fix all previous issues with version parsing (closes #121 and #147).