yarikoptic
commented
2 years ago ping on this
Closed yarikoptic closed 2 years ago
yarikoptic
commented
2 years ago ping on this
yarikoptic
commented
2 years ago 
adswa
commented
2 years ago I can reproduce this - the reason is a failure to merge branches with a merge conflict in .gitattributes:
adina@muninn in /tmp/datalad_temp_test_drop4q54t7kn on git:dl-test-branch
❱ git merge -m "merge incoming" --allow-unrelated-histories incoming-native
Auto-merging .gitattributes
CONFLICT (add/add): Merge conflict in .gitattributes
Automatic merge failed; fix conflicts and then commit the result.
adina@muninn in /tmp/datalad_temp_test_drop4q54t7kn on dl-test-branch+! (merge)
❱ git st 1 !
On branch dl-test-branch
You have unmerged paths.
(fix conflicts and run "git commit")
(use "git merge --abort" to abort the merge)
Unmerged paths:
(use "git add <file>..." to mark resolution)
both added: .gitattributes
no changes added to commit (use "git add" and/or "git commit -a")
adina@muninn in /tmp/datalad_temp_test_drop4q54t7kn on dl-test-branch+! (merge)
❱ git diff
diff --cc .gitattributes
index af926ef,16e6cfe..0000000
--- a/.gitattributes
+++ b/.gitattributes
@@@ -1,2 -1,2 +1,6 @@@
* annex.backend=MD5E
-**/.git* annex.largefiles=nothing
++<<<<<<< HEAD
+**/.git* annex.largefiles=nothing
++=======
++**/.git* annex.largefiles=nothing
++>>>>>>> incoming-nativeThe reason for this merge conflict seems to be a EOL in one of the files as indicated by viewing the binary diff between them:
From a quick glance, it seems the incoming-native branches .gitattribtues is created from cat-file -p'ing the main branch'es config:
# inherit the standard attributes to ensure uniform behavior
# across branches
(repo.pathobj / '.gitattributes').write_text(
repo.call_git(
['cat-file', '-p', '{}:.gitattributes'.format(main_branch)]))Maybe that (now started to) introduce(s) a newline?
adswa
commented
2 years ago Oh, it might have been introduced with https://github.com/datalad/datalad/commit/fa96e6a6fb497fc09a210d196a5b937a646b984b - since then, the .gitattributes file we write contains a new line. pathlib's write_text method on the other hand only learned to incorporate a newline in Python 3.10 (and we're still testing on and supporting starting from 3.7): https://docs.python.org/3/library/pathlib.html
adswa
commented
2 years ago Here's a patch that fixes it for me and shows that that's indeed the issue:
diff --git a/datalad_ukbiobank/init.py b/datalad_ukbiobank/init.py
index 5732a18..5ef16a0 100644
--- a/datalad_ukbiobank/init.py
+++ b/datalad_ukbiobank/init.py
@@ -125,7 +125,6 @@ class Init(Interface):
logger=lgr,
refds=ds.path,
)
if 'incoming' in incoming_branches and not force:
yield dict(
res,
@@ -155,7 +154,7 @@ class Init(Interface):
# across branches
(repo.pathobj / '.gitattributes').write_text(
repo.call_git(
- ['cat-file', '-p', '{}:.gitattributes'.format(main_branch)]))
+ ['cat-file', '-p', '{}:.gitattributes'.format(main_branch)]) + '\n')
# save to incoming branch, provide path to avoid adding untracked
# content
ds.save(
@@ -210,5 +209,5 @@ def _add_incoming_branch(name, incoming_branches, repo, batchfile):
# inherit the standard attributes to ensure uniform behavior
# across branches
(repo.pathobj / '.gitattributes').write_text(
- repo.call_git(['cat-file', '-p', 'incoming:.gitattributes']))
+ repo.call_git(['cat-file', '-p', 'incoming:.gitattributes']) + '\n')
repo.save(message='Apply standard Git attributes', paths=['.gitattributes'])pretty ugly hack. How can we make this prettier? The causing commit was introduced in 0.15.0 if I see it correctly, yet minimal datalad version declared in setup.cfg is 0.14. We could add the patch with a comment to make it pretty once we only support Python 3.10 and up and declare a dependency on datalad 0.15 or higher, or we need to sense if there is a newline and if so, add it?
yarikoptic
commented
2 years ago I would have created a helper ensure_endswith (worth datalad.utils. IMHO) like
def ensure_endswith(s: str, ending: str):
return s if s.endswith(ending) else s + endingand use it in both spots. We don't want to breed newlines indefinitely right? ;-)
adswa
commented
2 years ago worth
datalad.utils.IMHO
but a helper in core wouldn't make our lives easier, right? :D
yarikoptic
commented
2 years ago not ATM indeed, I just mentioned "hypothetically" worthwhile there. Meanwhile could be local to this repo
adswa
commented
2 years ago I created #85
yarikoptic
commented
2 years ago I think this particular one was addressed in datalad core. new one(s) to come, so freeing up the subject ;)
I with with the release of 0.16.x series. See https://github.com/datalad/datalad-extensions/actions?query=workflow%3Atest-datalad_ukbiobank-maint