datalad / git-annex

A non-official clone of git-annex established for DataLad purposes. No PRs will be merged, but could be used to test perspective git-annex patches. Official git-annex repository: https://git.kitenet.net/index.cgi/git-annex.git/
14 stars 3 forks source link

should environments be "auto-upgraded" #182

Closed yarikoptic closed 8 months ago

yarikoptic commented 8 months ago

so in https://github.com/datalad/git-annex/blob/master/clients/testannex.sh to have

conda activate testannex
conda env update --file environment.yml --prune
python testannex.py "$@"

or alike? (NB on openmind, we activate environment through the path, not by name)

jwodder commented 8 months ago

Two caveats:

  1. Currently, updates to the code in clients/ are deployed by testannex.py running git pull on the local repo right after startup, and the script then restarts if anything changed. If we go with your suggestion, then we should move the git pull from testannex.py to testannex.sh (placing it before the conda env update).

  2. This would not affect clients that don't use Conda & testannex.sh, which currently is just the smaug client, which will need to switch to Conda. We should probably also update clients/README.md to say that only Conda and testannex.sh should be used.

on openmind, we activate environment through the path, not by name

What exactly do you mean by this?

yarikoptic commented 8 months ago

What exactly do you mean by this?

we have conda activate ./condaenv/ not conda activate testannex. Reasoning is in the commit msg

    on openmind we store condaenv locally since precious on space

    although it is also in HOME -- I was just running out of space and kept
    creating it under /om2 but then decided finally to just make one right
    here
jwodder commented 8 months ago

Customizing testannex.sh will conflict with the ability to autoupdate testannex.sh.

yarikoptic commented 8 months ago

that's true, but there always might be some systems idiosyncracies. I think relying on git merge to merge them and fail in some rare cases of conflicts should be ok. WDYT?

jwodder commented 8 months ago

I'd be OK with that as long as we can configure git to error on merge conflicts and keep erroring on every git pull thereafter until a human intervenes to resolve the conflict.

yarikoptic commented 8 months ago
I think that is the default `git merge` behavior... ```shell ❯ datalad create test1 create(ok): /tmp/test1 (dataset) ❯ datalad clone test1 test1-clone install(ok): /tmp/test1-clone (dataset) ❯ echo 123 > test1/file && datalad save -d test1 add(ok): file (file) save(ok): . (dataset) action summary: add (ok: 1) save (ok: 1) ❯ echo 124 > test1-clone/file && datalad save -d test1-clone add(ok): file (file) save(ok): . (dataset) action summary: add (ok: 1) save (ok: 1) ❯ git -C test1-clone pull remote: Enumerating objects: 10, done. remote: Counting objects: 100% (10/10), done. remote: Compressing objects: 100% (7/7), done. remote: Total 8 (delta 0), reused 0 (delta 0), pack-reused 0 Unpacking objects: 100% (8/8), 811 bytes | 811.00 KiB/s, done. From ../test1 3372436..7ec3072 master -> origin/master 00fb544..ae073bf git-annex -> origin/git-annex hint: You have divergent branches and need to specify how to reconcile them. hint: You can do so by running one of the following commands sometime before hint: your next pull: hint: hint: git config pull.rebase false # merge hint: git config pull.rebase true # rebase hint: git config pull.ff only # fast-forward only hint: hint: You can replace "git config" with "git config --global" to set a default hint: preference for all repositories. You can also pass --rebase, --no-rebase, hint: or --ff-only on the command line to override the configured default per hint: invocation. fatal: Need to specify how to reconcile divergent branches. ❯ git -C test1-clone config pull.rebase false # merge ❯ git -C test1-clone pull CONFLICT (add/add): Merge conflict in file Automatic merge failed; fix conflicts and then commit the result. ❯ echo $? 1 ❯ git -C test1-clone pull error: Pulling is not possible because you have unmerged files. hint: Fix them up in the work tree, and then use 'git add/rm ' hint: as appropriate to mark resolution and make a commit. fatal: Exiting because of an unresolved conflict. ❯ echo $? 128 ```