OSX build fails

[yarikoptic]

https://github.com/datalad/git-annex/runs/4465340747?check_suite_focus=true

[24 of 24] Compiling Main             ( Build/Standalone.hs, Build/Standalone.o )
Linking Build/Standalone ...
./Build/Standalone tmp/build-dmg/git-annex.app/Contents/MacOS/bundle tmp/build-dmg/git-annex.app
cp: /usr/lib/libiconv.2.dylib: No such file or directory
installing /usr/local/opt/pcre2/lib/libpcre2-8.0.dylib as A
Standalone: library install failed
CallStack (from HasCallStack):
  error, called at ./Build/OSXMkLibs.hs:73:41 in main:Build.OSXMkLibs
installing /usr/lib/libiconv.2.dylib as B
make: *** [osxapp] Error 1

attn @joeyh @jwodder (since I don't know if that is env issue or annex code issue)

[jwodder]

The failure coincided with an upgrade of the macOS runner from 10.15 to 11.1. I believe git-annex (specifically, Build/OSXMkLibs.hs) is the one that needs to be updated.

[joeyh]

Looks like the dynamlic linker is listing library paths that somehow do not exist. I don't know what could cause this, but presumably something in the OS image being used is wrong.

-- see shy jo

[yarikoptic]

Just FTR: issue persists

[jwodder]

@joeyh Here's the reason for the error:

New in macOS Big Sur 11.0.1, the system ships with a built-in dynamic linker cache of all system-provided libraries. As part of this change, copies of dynamic libraries are no longer present on the filesystem. Code that attempts to check for dynamic library presence by looking for a file at a path or enumerating a directory will fail. Instead, check for library presence by attempting to dlopen() the path, which will correctly check for the library in the cache.

See also: this link.

I'm not familiar enough with git-annex's build procedure on macOS to know how it should be updated for this.

[yarikoptic]

Temporarily, could we still use old 10 osx env via runs-on like they instructed for upgrade to 11 https://github.blog/changelog/2021-08-16-github-actions-macos-11-big-sur-is-generally-available-on-github-hosted-runners/ ?

[yarikoptic]

IIRC sticking to oldest available also might be desired in the longer run to improve portability of the build

[joeyh]

Wow, of all the ways to break libraries...

Well, git-annex needs to build on current OSX going forward. If these special dynaamic libraries are so tangled up in the OS like this, that they're basically part of the dynamic linker, it probably does not make sense to try to bundle them in the dmg at all. Worst case, using the dmg on a version of OSX with an incompatible version of those libraries would fail at runtime.

Note that git-annex already omitted some OSX libraries/frameworks that were tightly coupled to the OS version.

[joeyh]

hopefully fixed in git-annex commit 8.20211231-11-gabeb7dcdd

[yarikoptic]

@joeyh do you think this would be a viable fix for build time or also for run time (i.e of a build from 10 ran on 11)? Filed related #98 -- with such changes between OSX versions I think we should add at least some basic testing of a built git-annex as shipped in the .dmg across a set of "supported" OSX releases.

[joeyh]

Compatability between OSX versions has always been ill-defined because it already had those other frameworks and system libraries that were not compatible and so are not included in the dmg. It will most likely be about as compatible in whatever direction as it was before, I guess.