Starlink init scripts should not set DYLD_LIBRARY_PATH

[smaret]

On MacOS Starlink init scripts (starlink.login and starlink.sh) set the DYLD_LIBRARY_PATH variable to point to Starlink installed libraries. This causes problems for programs installed with Fink. For example, Fink's Python version uses version 8.5.0 of TK library, which is installed under /sw/lib. If one set the DYLD_LIBRARY_PATH to include /star/lib in the path, Python attempts to use the TK version under /star/lib, and it crashes if the version is older than the one it was compiled with.

Indeed, if the binaries are linked properly, one should not need to set this variable, because the executable contains the full path to libraries.

[timj]

Ideally the Starlink applications would link using a relative path matching the way that application bundles work. I have made an attempt at hacking that but got stymied by Gnu libtool and associated build tools which are very old versions that have been hacked to support Fortran (@nxg is looking in to modernising those tools but it's a lot of work and we first need to get the Fortran patches accepted into autoconf).

Currently the build tools burn in complete paths when linking so if you aren't relocating from the build location then everything should work. The problem with "should" is that it seems that the CFITSIO build is non-standard because it doesn't include any path (no -R option) as it's using a configure script from the CFITSIO distribution. If you unset DYLD_LIBRARY_PATH things fail specifically because libcfitsio can't be found. Maybe @pwdraper can comment on that. So I think if cfitsio was fixed then at least DYLD_LIBRARY_PATH could be unset for a Starlink install that was not relocated.

The final usage of DYLD_LIBRARY_PATH relates to the Starjava JNI search path. The JNI library creation doesn't use -R so the Java applications can't find the locations to the libraries without help. I don't really know much about those builds although I imagine the existence of those paths in the environment wouldn't affect fink.

[timj]

@pwdraper Do you have any thoughts on the CFITSIO linking issue?

Just so I don't need to look it up again, the trick with the linker on OS X is to use "@rpath" in the path designation for -install_name. It would be nice if autoconf would do that directly rather than hard-coding the path with -install_name but I don't know if modern autoconfs do that. @nxg do you know?

[timj]

An alternative is to "fix up" the binaries after we've made them by running install_name_tool with the -add_rpath option. @MalcolmCurrie @grahambell We could do this when we ship a Starlink release. It's not needed for people doing private builds since they tend not to be interested in relocating their Starlink tree.

[nxg]

That sounds like the right thing to do. My recollection, from a little while ago, is that DYLD_LIBRARY_PATH is discouraged on OS X, and indeed dyld(1) suggests that it's to be used only for testing purposes.

[timj]

So first we have to get the build working without DYLD_LIBRARY_PATH (so fixing CFITSIO for example). @pwdraper can you sort out the CFITSIO issue?

Then for releases we have the script that adds the relative @rpth post-install. Maybe @grahambell can do the latter.

[pwdraper]

A possible option for CFITSIO is to use the code at https://github.com/sfabbro/cfitsio, those are patches for upgrading the build system so that libtool is used. Failing that we just need to pass in some LDFLAGS and get them used, that looks easy, not so sure where the flags would originate with a patched libtool being the main driver.

[timj]

@pwdraper Could the parent configure script in thirdparty/heasarc/cfitsio/ call the patched CFITSIO configure script with the necessary arguments?

[pwdraper]

If we can re-generate the patched build system with our autotools, then no need. The parent would just do the usual recursive call to a subdirectory and you'd get the right libtool.

[sfgraves]

Assuming nobody is going to be able to fix-up the build system to do this before the next Starlink release in Mid march, I'm going to see if I can implement the script that fixes up the paths afterwards.

After a very brief look for some info on paths on Mac and install_name_tool, I think I'll need a script that does:

1) add an rpath to the cfitisio .dylibs on the mac install. This will prevent DYLD_LIBRARY_PATH from needing to be set.

We'd need to use install_name_tool -id to change the path in the .dylibs to be relative to the starlink top-level directory.

Then use install_name_tool -add-rpath on each binary that links to cfitisio with the relative path to the libcitsio.dylib (using either \@loader_path or \@executable_path I guess?), And then use install_name_tool -change to use the @rpath in the binary.

2) If we want to make the MacOSX build relocatable, we'd need to have a script that repeats the same procedure on all the .dylibs and all the binaries.

( would this cause problems with libjpeg issues in issue #26 ?)

I'm going to see if I can get this working first for cfitisio.

[sfgraves]

I've commited a post-makeworld script (macos_postbuild_library_fixup.sh in commit c506e0155df4c13c8784ae01a94) that attempts to fixup all the binaries and dylibs shipped under $STARLINK_DIR/lib and $STARLINK_DIR/bin so that:

a) all dylibs have an id that uses @ rpath b) all the libraries that any dylib or binary attempts to load are at paths relative to the @ rpath c) all dylibs and libraries have their relative path to $STARLINK_DIR added as an rpath

This still requires testing, but if it works then this issue can be closed and DYLD_LIBRARY_PATH won't be required after running the script. It should also make the build relocatable.

[sfgraves]

The 2015A OSX release that we just shipped used the post-build fix up scripts to add rpaths to all the libraries and copy in missing dependency libraries (e.g. gfortran) that we don't build. It therefore does not set DYLD_LIBRARY_PATH.

Is there any hope of fixing up the build process to set the rpaths properly in the near future, or shall we continue using the post-build solution?

If we're going to use this post-build solution for a while I'd like to make it a little more automated (i.e. I had to hand edit the profile/login/cshrc scripts) and describe it properly in the README file.

[timj]

Fixing the build system is beyond my autoconf abilities. If @pwdraper had a Mac then we'd probably have it working by now :smile:

I'm getting a bit wound up by the JPEG issue. We should never build our own JPEG library on the Mac as it's always possible to use the system ImageIO framework. The trick is teaching configure that that is an option.

[sfgraves]

@grahambell did recently buy a book on autoconf, but I don't think he wants to do much that's OSX specific...

Do you know if the libJPEG shipped in the ImageIO framework will 'just work' if we try and build against it -- i.e. if we could get the autoconf system to pass the -framework ImageIO flag to the compiler would our code all build and work without changes, or would we have to re-write any of the actual code?