Can't compile when aerospike-client-c is compiled using luajit

[ultrabug]

When the C client has been built using the USE_LUAJIT = 1, the python driver fails to build with the following error.

x86_64-pc-linux-gnu-gcc -pthread -fPIC -I./aerospike-client-c/include -I./aerospike-client-c/include/ck -Isrc/include -I/usr/include/python2.7 -c src/main/predicates.c -o build/temp.linux-x86_64-2.7/src/main/predicates.o -std=gnu99 -g -Wall -fPIC -O1 -fno-common -fno-strict-aliasing -finline-functions -march=nocona -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D_GNU_SOURCE -rdynamic -DMARCH_x86_64
x86_64-pc-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,--as-needed -L. build/temp.linux-x86_64-2.7/src/main/aerospike.o build/temp.linux-x86_64-2.7/src/main/log.o build/temp.linux-x86_64-2.7/src/main/client/type.o build/temp.linux-x86_64-2.7/src/main/client/apply.o build/temp.linux-x86_64-2.7/src/main/client/close.o build/temp.linux-x86_64-2.7/src/main/client/connect.o build/temp.linux-x86_64-2.7/src/main/client/exists.o build/temp.linux-x86_64-2.7/src/main/client/exists_many.o build/temp.linux-x86_64-2.7/src/main/client/get.o build/temp.linux-x86_64-2.7/src/main/client/get_many.o build/temp.linux-x86_64-2.7/src/main/client/select_many.o build/temp.linux-x86_64-2.7/src/main/client/info_node.o build/temp.linux-x86_64-2.7/src/main/client/info.o build/temp.linux-x86_64-2.7/src/main/client/key.o build/temp.linux-x86_64-2.7/src/main/client/put.o build/temp.linux-x86_64-2.7/src/main/client/operate.o build/temp.linux-x86_64-2.7/src/main/client/query.o build/temp.linux-x86_64-2.7/src/main/client/remove.o build/temp.linux-x86_64-2.7/src/main/client/scan.o build/temp.linux-x86_64-2.7/src/main/client/select.o build/temp.linux-x86_64-2.7/src/main/client/admin.o build/temp.linux-x86_64-2.7/src/main/client/udf.o build/temp.linux-x86_64-2.7/src/main/client/sec_index.o build/temp.linux-x86_64-2.7/src/main/serializer.o build/temp.linux-x86_64-2.7/src/main/client/remove_bin.o build/temp.linux-x86_64-2.7/src/main/client/get_key_digest.o build/temp.linux-x86_64-2.7/src/main/client/lstack.o build/temp.linux-x86_64-2.7/src/main/client/lset.o build/temp.linux-x86_64-2.7/src/main/client/lmap.o build/temp.linux-x86_64-2.7/src/main/client/llist.o build/temp.linux-x86_64-2.7/src/main/key/type.o build/temp.linux-x86_64-2.7/src/main/key/apply.o build/temp.linux-x86_64-2.7/src/main/key/exists.o build/temp.linux-x86_64-2.7/src/main/key/get.o build/temp.linux-x86_64-2.7/src/main/key/put.o build/temp.linux-x86_64-2.7/src/main/key/remove.o build/temp.linux-x86_64-2.7/src/main/query/type.o build/temp.linux-x86_64-2.7/src/main/query/apply.o build/temp.linux-x86_64-2.7/src/main/query/foreach.o build/temp.linux-x86_64-2.7/src/main/query/results.o build/temp.linux-x86_64-2.7/src/main/query/select.o build/temp.linux-x86_64-2.7/src/main/query/where.o build/temp.linux-x86_64-2.7/src/main/scan/type.o build/temp.linux-x86_64-2.7/src/main/scan/foreach.o build/temp.linux-x86_64-2.7/src/main/scan/results.o build/temp.linux-x86_64-2.7/src/main/scan/select.o build/temp.linux-x86_64-2.7/src/main/lstack/type.o build/temp.linux-x86_64-2.7/src/main/lstack/lstack_operations.o build/temp.linux-x86_64-2.7/src/main/lset/type.o build/temp.linux-x86_64-2.7/src/main/lset/lset_operations.o build/temp.linux-x86_64-2.7/src/main/llist/type.o build/temp.linux-x86_64-2.7/src/main/llist/llist_operations.o build/temp.linux-x86_64-2.7/src/main/lmap/type.o build/temp.linux-x86_64-2.7/src/main/lmap/lmap_operations.o build/temp.linux-x86_64-2.7/src/main/conversions.o build/temp.linux-x86_64-2.7/src/main/policy.o build/temp.linux-x86_64-2.7/src/main/predicates.o ./aerospike-client-c/lib/libaerospike.a -L/usr/lib64 -lssl -lcrypto -lpthread -lm -lz -lrt -lpython2.7 -o build/lib.linux-x86_64-2.7/aerospike.so -Wl,-Bdynamic -fPIC
/usr/lib/gcc/x86_64-pc-linux-gnu/4.9.2/../../../../x86_64-pc-linux-gnu/bin/ld: ./aerospike-client-c/lib/libaerospike.a(lj_api.o): relocation R_X86_64_32S against `lj_obj_typename' can not be used when making a shared object; recompile with -fPIC
./aerospike-client-c/lib/libaerospike.a: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
error: command 'x86_64-pc-linux-gnu-gcc' failed with exit status 1

[psi-mankoski]

Hello --

In replicating your issue, we have encountered a few sub-issues, which we are planning to fix in forthcoming releases. That being said, from the build log snippet you sent, we have a few questions:

1). What is the distro. & version you are using? (Gentoo??)

2). What version of GCC are you using? (E.g., "gcc -dumpversion".)

3). Could you please tell us the exact commands you used to do this build? Specifically, according to the Python client "BUILD.md" file, the env. var. "NO_RESOLVE_C_CLIENT_DEP=True" needs to be set for "setup.py", otherwise the released version of the C client will be downloaded an used, which could result in the link error you reported.

When we made the changes that are not yet released, on our CentOS 6 system, the build succeeded. We can work with you to vet out those build script changes once we learn the answers to our 3 questions.

Thanks.

[ultrabug]

1). What is the distro. & version you are using? (Gentoo??)

Gentoo Linux yep, I found this issue while packaging the aerospike-client-python

2). What version of GCC are you using? (E.g., "gcc -dumpversion".)

4.9.2

3). Could you please tell us the exact commands you used to do this build? Specifically, according to the Python client "BUILD.md" file, the env. var. "NO_RESOLVE_C_CLIENT_DEP=True" needs to be set for "setup.py", otherwise the released version of the C client will be downloaded an used, which could result in the link error you reported.

Here is the full build.log : https://gist.github.com/ultrabug/c2894b5d8315471aaa99

When we made the changes that are not yet released, on our CentOS 6 system, the build succeeded. We can work with you to vet out those build script changes once we learn the answers to our 3 questions.

FYI, the Gentoo ebuild (source build "script") I wrote for this package : https://gitweb.gentoo.org/dev/ultrabug.git/tree/dev-python/aerospike/aerospike-1.0.42.ebuild

[psi-mankoski]

Thanks for the detailed info. ~~ Very helpful! From the top of the build log, I see it found "/usr/lib/libaerospike.a", presumably the one you built and installed with LuaJit enabled. The link line, however, shows "./aerospike-client-c/lib/libaerospike.a". Is that local library copy also the correct version? (And could you please send the build script & log for building that version of the Aerospike C client?) Will investigate further and get back to you.

[ultrabug]

I see it found "/usr/lib/libaerospike.a", presumably the one you built and installed with LuaJit enabled. The link line, however, shows "./aerospike-client-c/lib/libaerospike.a". Is that local library copy also the correct version?

No indeed, this is the one coming from the aerospike-client-c.sh. So I have changed the aerospike_c_prefix to /usr in the setup.py to make sure the build system uses the /usr/lib/libaerospike.a file but this gives me the same error result.

Sed to modify the c_prefix :

sed -e "s@aerospike_c_prefix = './aerospike-client-c'@aerospike_c_prefix = '/usr'@g" -i setup.py

Build error :

x86_64-pc-linux-gnu-gcc -shared -Wl,-O1 -Wl,--as-needed -O2 -pipe -march=nocona /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/aerospike.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/log.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/client/type.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/client/apply.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/client/close.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/client/connect.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/client/exists.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/client/exists_many.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/client/get.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/client/get_many.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/client/select_many.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/client/info_node.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/client/info.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/client/key.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/client/put.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/client/operate.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/client/query.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/client/remove.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/client/scan.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/client/select.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/client/admin.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/client/udf.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/client/sec_index.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/serializer.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/client/remove_bin.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/client/get_key_digest.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/client/lstack.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/client/lset.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/client/lmap.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/client/llist.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/key/type.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/key/apply.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/key/exists.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/key/get.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/key/put.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/key/remove.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/query/type.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/query/apply.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/query/foreach.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/query/results.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/query/select.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/query/where.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/scan/type.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/scan/foreach.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/scan/results.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/scan/select.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/lstack/type.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/lstack/lstack_operations.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/lset/type.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/lset/lset_operations.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/llist/type.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/llist/llist_operations.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/lmap/type.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/lmap/lmap_operations.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/conversions.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/policy.o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/temp.linux-x86_64-2.7/src/main/predicates.o /usr/lib/libaerospike.a -L/usr/lib64 -lssl -lcrypto -lpthread -lm -lz -lrt -lpython2.7 -o /dev/shm/portage/dev-python/aerospike-1.0.42/work/aerospike-1.0.42-python2_7/lib/aerospike.so
/usr/lib/gcc/x86_64-pc-linux-gnu/4.9.2/../../../../x86_64-pc-linux-gnu/bin/ld: /usr/lib/libaerospike.a(lj_api.o): relocation R_X86_64_32S against `lj_obj_typename' can not be used when making a shared object; recompile with -fPIC
/usr/lib/libaerospike.a: error adding symbols: Bad value
collect2: error: ld returned 1 exit status

And could you please send the build script & log for building that version of the Aerospike C client?

I have updated the gist with the C client build.log : https://gist.github.com/ultrabug/c2894b5d8315471aaa99

[ultrabug]

Quick note : just saw that the C client build.log was for a lua compilation, updated with luajit enabled build.log

[psi-mankoski]

Thanks for the build details ~~ They are a big help for reproducing the issue.

[marknaero]

With pip (>=19.0) and latest python client (>=3.8.0), you should be able to install on linux without needing to compile or install any other dependencies anymore.