Symbol not found: _pbbufr_ (Mac OS Mojave)

[Akramz]

First of all, Thank you for your valuable work, I always use your library to decode BUFR.

My Problem happened after i upgraded my OS to Mojave, at first a noticed that the library stopped importing correctly, producing this error:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-2-a9afcbee3214> in <module>()
      9 import skimage.measure
     10 from pybufr_ecmwf.raw_bufr_file import RawBUFRFile
---> 11 from pybufr_ecmwf.bufr_interface_ecmwf import BUFRInterfaceECMWF
     12 import scipy
     13 import sklearn

~/.local/lib/python3.6/site-packages/pybufr_ecmwf/bufr_interface_ecmwf.py in <module>()
     50     print('import of ecmwfbufr failed.')
     51     print('sys.path = ', sys.path)
---> 52     raise e
     53 
     54 from . import ecmwfbufr_parameters

~/.local/lib/python3.6/site-packages/pybufr_ecmwf/bufr_interface_ecmwf.py in <module>()
     46 # import the raw wrapper interface to the ECMWF BUFR library
     47 try:
---> 48     from . import ecmwfbufr
     49 except ImportError as e:
     50     print('import of ecmwfbufr failed.')

ImportError: dlopen(/Users/mohamedakramzaytar/.local/lib/python3.6/site-packages/pybufr_ecmwf/ecmwfbufr.cpython-36m-darwin.so, 2): Library not loaded: /usr/local/opt/gcc/lib/gcc/7/libgfortran.4.dylib
  Referenced from: /Users/mohamedakramzaytar/.local/lib/python3.6/site-packages/pybufr_ecmwf/ecmwfbufr.cpython-36m-darwin.so
  Reason: image not found

I then proceeded to reinstall it following the usual steps described in here, it gave back this error:

a python wrapper to the ECMWF BUFR library has been generated
compilation of library wrapper finished
extracting library constants
Traceback (most recent call last):
  File "setup.py", line 414, in <module>
    provides=["pybufr_ecmwf"])
  File "/anaconda3/envs/indaba/lib/python3.6/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/anaconda3/envs/indaba/lib/python3.6/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/anaconda3/envs/indaba/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "setup.py", line 123, in run
    _build.run(self)
  File "/anaconda3/envs/indaba/lib/python3.6/distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/anaconda3/envs/indaba/lib/python3.6/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/anaconda3/envs/indaba/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/anaconda3/envs/indaba/lib/python3.6/distutils/command/build_ext.py", line 339, in run
    self.build_extensions()
  File "/anaconda3/envs/indaba/lib/python3.6/distutils/command/build_ext.py", line 448, in build_extensions
    self._build_extensions_serial()
  File "/anaconda3/envs/indaba/lib/python3.6/distutils/command/build_ext.py", line 473, in _build_extensions_serial
    self.build_extension(ext)
  File "setup.py", line 322, in build_extension
    ibi.build()
  File "/Users/mohamedakramzaytar/libs/Downloads/pybufr-ecmwf/build_interface.py", line 893, in build
    self.extract_constants()
  File "/Users/mohamedakramzaytar/libs/Downloads/pybufr-ecmwf/build_interface.py", line 1985, in extract_constants
    import ecmwfbufr
ImportError: dlopen(/Users/mohamedakramzaytar/libs/Downloads/pybufr-ecmwf/build/lib.macosx-10.7-x86_64-3.6/pybufr_ecmwf/ecmwfbufr.cpython-36m-darwin.so, 2): Symbol not found: _pbbufr_
  Referenced from: /Users/mohamedakramzaytar/libs/Downloads/pybufr-ecmwf/build/lib.macosx-10.7-x86_64-3.6/pybufr_ecmwf/ecmwfbufr.cpython-36m-darwin.so
  Expected in: flat namespace
 in /Users/mohamedakramzaytar/libs/Downloads/pybufr-ecmwf/build/lib.macosx-10.7-x86_64-3.6/pybufr_ecmwf/ecmwfbufr.cpython-36m-darwin.so

After running tests:

Ran 65 tests in 6.096s

FAILED (failures=32, errors=31)
removing symlink pybufr_ecmwf
renaming pybufr_ecmwf.renamed to pybufr_ecmwf
just before exit: success =  False

Is there a way to fix this ?

[jdkloe]

Thanks for reporting your issue. Since the problems started when you did an upgrade, the most likely cause is that your python and c compiler where not updated at the same time. Compiled python modules like pybufr-ecmwf really need to be compiled by a c-compiler (and associated fortran compiler) that was the exact same one as the one that was used to compile the python interpreter itself. For MacOS the only way I could get it working was to use homebrew to install both python and a matching gcc/gfortran combination. So could you try this on your side? i.e. if needed reinstall homebrew compatible with Mojave. Then re-install the latest python, the latest gcc/gfortran, both from homebrew, and then rebuild the pybufr-ecmwf module? What happens in this case?

[Akramz]

Thank you for answering,

I've solved the error by installing GCC-8.2 from source and used it to install pybufr_ecmwf.

Other: I got a few errors when I tried re-doing the installation steps after updating brew, when running: brew tap Homebrew/python and brew install homebrew/python/numpy --with-python3 They Give back: Error: homebrew/python was deprecated. This tap is now empty as all its formulae were migrated.

Again, thank you very much for your efforts.

[jdkloe]

Hi, thanks for reporting back. So to help other users, could you explain how you solved the brew error? Or you did not and just ignored it? From the error message I get the impression that there has just been a name-change (but I cannot test it, since I have no Mac system at hand for testing).

[Akramz]

I'm working with Conda environments all the time, and python uses anaconda's python path by default (even with no environment activated), so I just ignored this error and it works fine.