resolves symlinks to .git/annex/-ed content

[yarikoptic]

user reported that when opening a git-annex'ed file, mricrogl shows the filename of the resolved file which is a long git-annex key:

I wonder if there could be an option to avoid resolving symlinks either globally or for specific files? Thanks in advance!

[neurolabusc]

@yarikoptic can you provide a simple demo of this issue. Is the filename provided by a Python script running inside MRIcroGL, or from an external source (drag and drop, command line call). I wonder if it is the operating system itself that is resolving the filenames, hiding the symlink name from the user and MRIcroGL. macOS graphical applications must live in a sandbox, and the OS tends to be pretty explicit about filenames to protect the user from phishing exploits.

[yarikoptic]

• install datalad

datalad install ///openneuro/ds000001
cd ds000001
datalad get sub-0[12]/anat/*T1w*nii.gz
mricrogl sub-0[12]/anat/*T1w*nii.gz

on OSX or linux (on windows you would not get symlinks) although could not try with my half-built mricrogl, but I believe it would reproduce . May be @jungheejung would provide a screenshot later on from OSX

[yarikoptic]

ha -- apparently the reproducer I had in mind didn't reproduce. @jungheejung reproduced by

• open mricrogl in finder
• it came up with mni152 already
• then clicked on "Add overlay" in the menu, and selected one of those files from the example above -- and she reproduced the image.

[neurolabusc]

@yarikoptic I am unable to replicate on my ARM based M1 MacBook. I can not install datalad. The simple brew install datalad failed which suggested I should run brew install --build-from-source datalad which failed and suggested I should try brew install --build-from-source git-annex which seems to have been stuck at the same point for a tremendously long time:

> brew install --build-from-source git-annex
==> Downloading https://ghcr.io/v2/homebrew/core/ca-certificates/manifests/2022-
Already downloaded: /Users/chrisrorden/Library/Caches/Homebrew/downloads/982f0412214a77590591ee391a99d93420da762beb3dfd0da08e2937777001b8--ca-certificates-2022-03-29.bottle_manifest.json
==> Downloading https://ghcr.io/v2/homebrew/core/ca-certificates/blobs/sha256:e7
Already downloaded: /Users/chrisrorden/Library/Caches/Homebrew/downloads/cca35bb4b10c436503180260d25e33bb53db60f6aee19523e4b153a697d99e84--ca-certificates--2022-03-29.all.bottle.tar.gz
==> Downloading https://ghcr.io/v2/homebrew/core/libgpg-error/manifests/1.44
Already downloaded: /Users/chrisrorden/Library/Caches/Homebrew/downloads/8eaec8fb52da4563f759cd6e19fb55fd408194fb6dac75b6550e3eddd1fe3ad5--libgpg-error-1.44.bottle_manifest.json
==> Downloading https://ghcr.io/v2/homebrew/core/libgpg-error/blobs/sha256:4049a
Already downloaded: /Users/chrisrorden/Library/Caches/Homebrew/downloads/f28f16bd93d02248f857deedab32f3bf9633fe1f81e85f5a013461ab3c0b3a65--libgpg-error--1.44.arm64_big_sur.bottle.tar.gz
==> Downloading https://ghcr.io/v2/homebrew/core/libgcrypt/manifests/1.10.1
Already downloaded: /Users/chrisrorden/Library/Caches/Homebrew/downloads/58a7ae87a8cba4e507dbf53db7e4fecb0f5b89fcfbff5b6bedf74b25cf8b0390--libgcrypt-1.10.1.bottle_manifest.json
==> Downloading https://ghcr.io/v2/homebrew/core/libgcrypt/blobs/sha256:3350e668
Already downloaded: /Users/chrisrorden/Library/Caches/Homebrew/downloads/5af7cb91486125271dd724b0357c25c0d609d47e70bce3bced7a3bc7eb34506f--libgcrypt--1.10.1.arm64_big_sur.bottle.tar.gz
==> Downloading https://ghcr.io/v2/homebrew/core/libmagic/manifests/5.41
Already downloaded: /Users/chrisrorden/Library/Caches/Homebrew/downloads/7d4d01d42b6a3eb0495ad4c1adcc39ba2f866cbd3a0cc85bb6c4f541d9870cf2--libmagic-5.41.bottle_manifest.json
==> Downloading https://ghcr.io/v2/homebrew/core/libmagic/blobs/sha256:b851d2364
Already downloaded: /Users/chrisrorden/Library/Caches/Homebrew/downloads/ddf57ace8947583094e6d7c612adbb4eea81ed38ca545456102d39ecb808733a--libmagic--5.41.arm64_big_sur.bottle.tar.gz
==> Downloading https://hackage.haskell.org/package/git-annex-10.20220322/git-an
######################################################################## 100.0%
Warning: Your Xcode (12.5.1) is outdated.
Please update to Xcode 13.2.1 (or delete it).
Xcode can be updated from the App Store.

==> Installing dependencies for git-annex: ca-certificates, libgpg-error, libgcrypt and libmagic
==> Installing git-annex dependency: ca-certificates
==> Pouring ca-certificates--2022-03-29.all.bottle.tar.gz
==> Regenerating CA certificate bundle from keychain, this may take a while...
🍺  /opt/homebrew/Cellar/ca-certificates/2022-03-29: 3 files, 211.6KB
==> Installing git-annex dependency: libgpg-error
==> Pouring libgpg-error--1.44.arm64_big_sur.bottle.tar.gz
🍺  /opt/homebrew/Cellar/libgpg-error/1.44: 27 files, 1MB
==> Installing git-annex dependency: libgcrypt
==> Pouring libgcrypt--1.10.1.arm64_big_sur.bottle.tar.gz
🍺  /opt/homebrew/Cellar/libgcrypt/1.10.1: 23 files, 3.3MB
==> Installing git-annex dependency: libmagic
==> Pouring libmagic--5.41.arm64_big_sur.bottle.tar.gz
🍺  /opt/homebrew/Cellar/libmagic/5.41: 341 files, 8.6MB
==> Installing git-annex
==> cabal v2-update
==> cabal v2-install --jobs=8 --max-backjumps=100000 --install-method=copy --ins

I suggest we defer this issue a bit until the datalad team supports the latest Macs. I do think that this is not an issue with MRIcroGL, but rather the way the operating system resolves names for the user.

[yarikoptic]

Thanks for trying ! Unfortunately indeed there is a problem building git-annex on M1 https://github.com/datalad/datalad/issues/5701 and thus we lack builds for it at https://github.com/datalad/git-annex/issues/61 . I bet we could easily reproduce it without involving git-annex/datalad by simply symlinking some file manually and seeing what name mricrogl would use (original one of symlink or the target one it points to). I will try in our next interactive session with @jungheejung .

[yarikoptic]

BTW @neurolabusc -- rosetta is the quick workaround for now on how to make git-annex available on M1. Some instructions are at https://github.com/fraimondo/csguide-hands-on#problem-3--git-annex-on-m1-mac and our datalad-installer would install rosetta on such beasts. I have tested on our m1 instance -- seems to work nicely.

[neurolabusc]

@yarikoptic I have not forgotten this. I am still unable to install the datalad on my computer.

$brew install --build-from-source git-annex
...
[490 of 639] Compiling Command.Smudge   ( Command/Smudge.hs, dist/build/git-annex/git-annex-tmp/Command/Smudge.o, dist/build/git-annex/git-annex-tmp/Command/Smudge.dyn_o )

<no location info>: error:
    Warning: Couldn't figure out LLVM version!
             Make sure you have installed LLVM between [9 and 13)
ghc: could not execute: opt
cabal: Failed to build git-annex-10.20220822. See the build log above for details.
...
$clang --version
Apple clang version 12.0.5 (clang-1205.0.22.11)

[yarikoptic]

what if you do not try to --build-from-source ? IIRC it was just a matter of /usr/sbin/softwareupdate --install-rosetta --agree-to-license and then installing .dmg. At least that is what I did and what is described in that URL I referred and I believe what https://github.com/datalad/datalad-installer/blob/master/src/datalad_installer.py would do for you if you run it as python datalad_installer.py git-annex --method=dmg --path=PATH_TO_DMG where dmg you can get from https://github.com/datalad/git-annex/releases/tag/10.20220822

[neurolabusc]

@yarikoptic what .dmg are you referring to? The installer does not include a dmg file?

Seems like I must install from source:

brew install  git-annex 
Running `brew update --auto-update`...
==> Auto-updated Homebrew!
Updated 2 taps (homebrew/core and homebrew/cask).
==> New Formulae
c               gator           hof             licenseplist    taplo
==> New Casks
apifox       beast2       doll         keet         macforge     postman-cli
apipost      bluebubbles  douyin       lectrote     notesnook    xprocheck

You have 13 outdated formulae installed.
You can upgrade them with brew upgrade
or list them with brew outdated.

Error: git-annex: no bottle available!

[neurolabusc]

@yarikoptic what .dmg are you referring to? The installer does not include a dmg file?

Seems like I must install from source:

brew install  git-annex 
Running `brew update --auto-update`...
==> Auto-updated Homebrew!
Updated 2 taps (homebrew/core and homebrew/cask).
==> New Formulae
c               gator           hof             licenseplist    taplo
==> New Casks
apifox       beast2       doll         keet         macforge     postman-cli
apipost      bluebubbles  douyin       lectrote     notesnook    xprocheck

You have 13 outdated formulae installed.
You can upgrade them with brew upgrade
or list them with brew outdated.

Error: git-annex: no bottle available!

[yarikoptic]

you can download dmg from https://github.com/datalad/git-annex/releases/tag/10.20220822 . I believe that datalad_installer git-annex -m datalad/git-annex:release would do that for you.

[neurolabusc]

Looks like this installs git-annex but

python datalad_installer.py git-annex --method=dmg --path=./git-annex_10.20220822_x64.dmg
2022-09-07T09:54:54-0400 [INFO    ] datalad_installer Writing environment modifications to /var/folders/4p/cxb1ys0s4wz6dh3rtj22pq8w0000gn/T/dl-env-p0qp2_l4.sh
2022-09-07T09:54:54-0400 [INFO    ] datalad_installer Installing git-annex via dmg
2022-09-07T09:54:54-0400 [INFO    ] datalad_installer Path: git-annex_10.20220822_x64.dmg
2022-09-07T09:54:54-0400 [INFO    ] datalad_installer M1 Mac detected; installing Rosetta
2022-09-07T09:54:54-0400 [INFO    ] datalad_installer Running: /usr/sbin/softwareupdate --install-rosetta --agree-to-license
By using the agreetolicense option, you are agreeing that you have run this tool with the license only option and have read and agreed to the terms.
If you do not agree, press CTRL-C and cancel this process immediately.
2022-09-07 09:54:55.204 softwareupdate[7283:310020] Package Authoring Error: 012-41532: Package reference com.apple.pkg.RosettaUpdateAuto is missing installKBytes attribute
Install of Rosetta 2 finished successfully
2022-09-07T09:54:56-0400 [INFO    ] datalad_installer Running: hdiutil attach git-annex_10.20220822_x64.dmg
expected   CRC32 $DEB97C73
/dev/disk4              GUID_partition_scheme           
/dev/disk4s1            Apple_APFS                      
/dev/disk5              EF57347C-0000-11AA-AA11-0030654 
/dev/disk5s1            41504653-0000-11AA-AA11-0030654 /Volumes/git-annex
2022-09-07T09:54:57-0400 [INFO    ] datalad_installer Running: rsync -a /Volumes/git-annex/git-annex.app /Applications/
2022-09-07T09:54:57-0400 [INFO    ] datalad_installer Running: hdiutil detach /Volumes/git-annex/
"disk4" ejected.
2022-09-07T09:54:57-0400 [INFO    ] datalad_installer git-annex is now installed at /Applications/git-annex.app/Contents/MacOS/git-annex
dyld: Library not loaded: @executable_path/B
  Referenced from: /Applications/git-annex.app/Contents/MacOS/bundle/git-annex
  Reason: no suitable image found.  Did find:
    /Applications/git-annex.app/Contents/MacOS/bundle/B: code signature invalid for '/Applications/git-annex.app/Contents/MacOS/bundle/B'

    /Applications/git-annex.app/Contents/MacOS/bundle/B: code signature invalid for '/Applications/git-annex.app/Contents/MacOS/bundle/B'

    /Applications/git-annex.app/Contents/MacOS/bundle/B: stat() failed with errno=1
2022-09-07T09:54:57-0400 [ERROR   ] datalad_installer `/Applications/git-annex.app/Contents/MacOS/git-annex --help` command failed!

A git-annex.app file is created in my Application folder when the dmg is installed either by the datalad_installer or clicking on the dmg icon. However, in both case the executable fails to run:

/Applications/git-annex.app/Contents/MacOS/git-annex
dyld: Library not loaded: @executable_path/B
  Referenced from: /Applications/git-annex.app/Contents/MacOS/bundle/git-annex
  Reason: no suitable image found.  Did find:
    /Applications/git-annex.app/Contents/MacOS/bundle/B: code signature invalid for '/Applications/git-annex.app/Contents/MacOS/bundle/B'

    /Applications/git-annex.app/Contents/MacOS/bundle/B: code signature invalid for '/Applications/git-annex.app/Contents/MacOS/bundle/B'

    /Applications/git-annex.app/Contents/MacOS/bundle/B: stat() failed with errno=1
zsh: abort      /Applications/git-annex.app/Contents/MacOS/git-annex

[neurolabusc]

I think this is relevant. No override appears on theSecurity and Privacy Systems Preferences panel.

While one user reported that --build-from-source worked for them, it is noted that compiler version matters and it did not work for me with clang 12.0.5 (others suggest using LLVM/Clang 12 over 13).

[yarikoptic]

I think this is relevant. No override appears on theSecurity and Privacy Systems Preferences panel.

did that resolve the issue so that you could run installed git annex or not?

[neurolabusc]

No, installing the dmg leads to the no suitable image found error. Attempting to install from source leads to a ghc: could not execute: opt, presumably since the Apple Clang/LLVM installation does not include that optimizer.

[yarikoptic]

Hi @neurolabusc - was it addressed somehow?

[neurolabusc]

I do not have experience with datalad to resolve this. MRIcroGL is reporting the file name provided by the file system. Feel free to provide a PR if you wish.