Build issue on OS X

[jrs65]

I get a build issue when trying to get this going on OS X.

$ python setup.py install
...
ld: warning: directory not found for option '-L/opt/local/lib'
duplicate symbol _bshuf_H5Filter in:
    build/temp.macosx-10.9-x86_64-2.7/bitshuffle/h5.o
    build/temp.macosx-10.9-x86_64-2.7/src/bshuf_h5filter.o
ld: 1 duplicate symbol for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: command 'gcc' failed with exit status 1

Any ideas what's going on?

[telegraphic]

Hey -- I think this is a libc vs. libstdc issue, e.g.: http://stackoverflow.com/questions/19637164/c-linking-error-after-upgrading-to-mac-os-x-10-9-xcode-5-0-1

If so, the fix would be to edit your ~/.bash_profile and add: export CFLAGS='-stdlib=libstdc++' export CXXFLAGS='-stdlib=libstdc++'

Then open a new terminal (or source ~/.bash_profile), and then try to compile.

[jrs65]

I've looked into this some more. This fix works, but weirdly not for the reasons given.

As there's no C++ actually being compiled here, it doesn't actually matter what CFLAGS is set to. In fact just setting it to be empty with:

export CFLAGS=''

works completely fine and gets rid of the errors. Looking a little more closely at the differences in compilation in the compilation lines, you see that the clang command changes from:

clang -fno-strict-aliasing -fno-common -dynamic -I/usr/local/include -I/usr/local/opt/sqlite/include -DNDEBUG -g -fwrapv -O3 -Wall ...

to

clang -DNDEBUG -g -fwrapv -O3 -Wall ...

i.e. that setting CFLAGS is overriding some default setting. The one that looks suspicious is the -fno-common one, and indeed that is the problem. If you try:

export CFLAGS='-fno-common'

you find that the build breaks again.

Also, from the gcc docs (the same option is true for clang)

-fno-common In C code, controls the placement of uninitialized global variables. Unix C compilers have traditionally permitted multiple definitions of such variables in different compilation units by placing the variables in a common block. This is the behavior specified by -fcommon, and is the default for GCC on most targets. On the other hand, this behavior is not required by ISO C, and on some targets may carry a speed or code size penalty on variable references. The -fno-common option specifies that the compiler should place uninitialized global variables in the data section of the object file, rather than generating them as common blocks. This has the effect that if the same variable is declared (without extern) in two different compilations, you get a multiple-definition error when you link them. In this case, you must compile with -fcommon instead. Compiling with -fno-common is useful on targets for which it provides better performance, or if you wish to verify that the program will work on other systems that always treat uninitialized variable declarations this way.

Looking at the source, that's exactly what is happening

H5Z_class_t bshuf_H5Filter[1];

will be defined in both object files. The solution here is to probably define bshuf_H5Filter as extern. I'll have a quick look into it.

[ranaivomahaleo]

Confirmed. Let us consider the file src/bshuf_h5filter.h file. Changing H5Z_class_t bshuf_H5Filter[1]; to extern H5Z_class_t bshuf_H5Filter[1]; solved this issue.

[kiyo-masui]

Great! I'll push a tag for this as soon as I can.