Darwin build failing without libstdc++

[mattgibbs]

Apple is starting to phase out libstdc++ on macOS, instead relying on libc++. With very recent versions of the Xcode Command Line Build Tools, libstdc++ isn't even available. epicscorelibs specifically asks for libstdc++ in src/python/epicscorelibs/config.py:

    if OS_CLASS=='Darwin':
        build['CPPFLAGS'] += [('darwin', None)]
        build['CXXFLAGS'] += ['-std=c++98', '-stdlib=libstdc++']

...which causes the build to fail if libstdc++ isn't available.

I can confirm that if you remove this line, the library will build again. I am not qualified to know if the EPICS core libs really are compatible with libc++ though. For what its worth, the p4p RPC examples do work with epicscorelibs built against libc++.

[mdavidsaver]

I agree that this is not an optimal. I had quite some difficulty finding a recipe which worked. With other combinations I was seeing strange linking errors in the travis-ci builds. Without having a mac myself, troubleshooting this because overly frustrating, so I stopped when I found a working recipe.

Are you interested/able to help sort out any errors which result from switching to c++11 and/or libc++?

[mdavidsaver]

FYI. travis-ci builders currently claims

Runtime kernel version: 17.4.0
...
ProductName:    Mac OS X
ProductVersion: 10.13.3
BuildVersion:   17D102

[mattgibbs]

Yeah, I think MacOS 10.14 is when libstdc++ disappeared. "Overly frustrating" is probably an understatement for this whole situation. I'm happy to help as problems reveal themselves, although I know very little about EPICS internals, so I might be pretty slow at first.

[mdavidsaver]

I tried removing any specification of c++ version or library. ab3d324705767bd1a6fae5c267dc94326758b6f8 The result is

ERROR: Failure: ImportError (dlopen(/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/p4p/_p4p.so, 2): Symbol not found: __ZN4pvac13ClientChannel3getEPNS0_11GetCallbackENSt3__110shared_ptrIKN5epics6pvData11PVStructureEEE
  Referenced from: /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/p4p/_p4p.so
  Expected in: flat namespace
 in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/p4p/_p4p.so)

The P4P build which fails

https://travis-ci.org/mdavidsaver/p4p/jobs/565252497

The corresponding epicscorelibs build

https://travis-ci.org/mdavidsaver/epicscorelibs/jobs/565234381

My very first question is what the missing symbol is when unmangled:

__ZN4pvac13ClientChannel3getEPNS0_11GetCallbackENSt3__110shared_ptrIKN5epics6pvData11PVStructureEEE

I would then suggest to fetch down the epicscorelibs build binaries and see what similar symbol is actually provided. From past experiences w/ Linux, I suspect the difference will involve class shared_ptr, which will be in different namespaces depending on which language version and library are selected.

[mdavidsaver]

I know very little about EPICS internals, so I might be pretty slow at first.

I suspect that neither EPICS, nor python, internals play any role here. Rather, I think this will be about how the various compiler flags effect c++ ABI.

[mdavidsaver]

I expect that the symbol in question will unmangle as something like:

pvac::ClientChannel::get(pvac::ClientChannel::GetCallback*, std::shared_ptr<epics::pvData::PVStructure const>)

With some variation to std::shared_ptr.

[mdavidsaver]

@mattgibbs I'd like to push out an update to epicscorelibs with the recently released Base 7.0.3. Do you plan to look into this issue soon? Otherwise, can you say whether the currently produced binary wheels for Mac are usable (and actually useful)?

I'm could switch back to -std=c++98 for the time being, or skip uploading wheel builds for Mac altogether.

[mattgibbs]

@mdavidsaver I just tried the binary wheel for Python 2.7 on OS X 10.14 and it worked fine. There are currently no binaries for Python 3.7, so I can't test that, although I assume it will work too. I'd say keep on uploading the mac builds - probably >90% of users aren't building from source anyway.

[mdavidsaver]

Ok. I'll revert ab3d324705767bd1a6fae5c267dc94326758b6f8

no binaries for Python 3.7

I'm planning to add this with the next update.

[mdavidsaver]

Another variable in the CI build process is the presence of two variations of installer with differing suffix -macosx10.6.pkg vs. -macosx10.9.pkg. So far I'm using the 10.6 variety, though I don't understand the significance of this choice. cf. https://www.python.org/ftp/python/3.7.4/

[mdavidsaver]

I've gotten stuck with the py3.7 builds for osx. I suspect that the platform tag I use for the <=3.6 builds (eg. cp36-cp36m-macosx_10_6_intel.macosx_10_6_x86_64) is not correct for 3.7 as it appears that pip does not choose it from pypi.org.

Does pip install --pre --only-binary :all: epicscorelibs work for anyone w/ python 3.7 on osx?

My reading of https://github.com/MacPython/wiki/wiki/Spinning-wheels suggests to me that cp37-cp37m-macosx_10_6_intel.macosx_10_6_x86_64 should work, but that seems not to be the case.

[hhslepicka]

$ python --version
Python 3.7.4

$ pip install --pre --only-binary :all: epicscorelibs
Collecting epicscorelibs
  ERROR: Could not find a version that satisfies the requirement epicscorelibs (from versions: none)
ERROR: No matching distribution found for epicscorelibs

[mdavidsaver]

Some progress. One mistake I was making was not also passing -std=c++11 -stdlib=libc++ to the linker, which of course needs to select a c++ library to link against.

[mdavidsaver]

I think I have sorted through the build problems, OSX and otherwise. When you can please retest both binary and source (re)build. You'll need to give pip --pre to pull in p4p version 3.3.2a1

Note that you may see a unittest crashing (p4p.test.test_sharedpv.TestFirstLast). This is a regression from a fix added in 3.3.1 which I'm still working on.

[hhslepicka]

It worked fine for me when using the binary distribution and installing using the source build. Python 3.7.3 and macOS 10.14.6 👍

[mdavidsaver]

I think this is resolved as of epicscorelibs 7.0.3.99.1.0 and p4p 3.3.2. Please reopen if further issues are encountered.