Can't compile as dynamic library in linux.

[Hou-Xiaoxuan]

Use PIPY_SHARED to complied as dynamic library. Reproduction method：

[I] user@mac ~/w/p/l/p/build ((c009821d))> git log -1
commit c009821dd1d4f0e0cfd7fda0720e6f0456e4525f (HEAD, main)
Author: pajama-coder <pajamacoder@flomesh.io>
Date:   Thu May 23 20:18:54 2024 +0800

    [fix] Crash in non-shared Quota due to missing of a null-check

[I] user@mac ~/w/p/l/p/build ((c009821d))> cmake -DPIPY_SHARED=ON ..
-- The C compiler identification is GNU 14.1.1
-- The CXX compiler identification is GNU 14.1.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Deprecation Warning at deps/yajl-2.1.0/CMakeLists.txt:15 (CMAKE_MINIMUM_REQUIRED):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.
  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.
......
-- Build files have been written to: /pipy-rs/libs/pipy/build

[I] user@mac ~/w/p/l/p/build ((c009821d))> make -j4
[  0%] Building C object deps/brotli-1.0.9/CMakeFiles/brotlicommon-static.dir/c/common/constants.c.o
[  0%] Built target OpenSSL
[  0%] Built target GenVer
[  0%] Building C object deps/yajl-2.1.0/src/CMakeFiles/yajl_s.dir/yajl.c.o
......
[ 12%] Building C object deps/yajl-2.1.0/src/CMakeFiles/yajl.dir/yajl_parser.c.o
[ 12%] Building C object deps/zlib-1.3/CMakeFiles/zlibstatic.dir/gzread.c.o
pipy-rs/libs/pipy/deps/yajl-2.1.0/src/yajl_parser.c: In function ‘yajl_do_parse’:
pipy-rs/libs/pipy/deps/yajl-2.1.0/src/yajl_parser.c:334:24: warning: this statement may fall through [-Wimplicit-fallthrough=]
  334 |                     if (yajl_bs_current(hand->stateStack) ==
      |                        ^
pipy-rs/libs/pipy/deps/yajl-2.1.0/src/yajl_parser.c:347:17: note: here
  347 |                 case yajl_tok_colon:
      |                 ^~~~
pipy-rs/libs/pipy/deps/yajl-2.1.0/src/yajl_parser.c:405:24: warning: this statement may fall through [-Wimplicit-fallthrough=]
  405 |                     if (yajl_bs_current(hand->stateStack) ==
      |                        ^
pipy-rs/libs/pipy/deps/yajl-2.1.0/src/yajl_parser.c:414:17: note: here
  414 |                 default:
      |                 ^~~~~~~
[ 12%] Building C object deps/zlib-1.3/CMakeFiles/zlibstatic.dir/gzwrite.c.o
......
[ 62%] Building CXX object CMakeFiles/pipy.dir/src/api/algo.cpp.o
In file included from pipy-rs/libs/pipy/src/pjs/pjs.hpp:29,
                 from pipy-rs/libs/pipy/src/event.hpp:29,
                 from pipy-rs/libs/pipy/src/filter.hpp:29,
                 from pipy-rs/libs/pipy/src/admin-link.hpp:29,
                 from pipy-rs/libs/pipy/src/admin-link.cpp:26:
pipy-rs/libs/pipy/src/pjs/types.hpp:1206:43: error: redefinition of ‘bool __tls_guard’
 1206 | template<class T> thread_local Ref<Class> ClassDef<T>::m_c;
      |                                           ^~~~~~~~~~~
pipy-rs/libs/pipy/src/pjs/types.hpp:1206:43: note: ‘bool __tls_guard’ previously declared here
pipy-rs/libs/pipy/src/pjs/types.hpp:1206: confused by earlier errors, bailing out
make[2]: *** [CMakeFiles/pipy.dir/build.make:76: CMakeFiles/pipy.dir/src/admin-link.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
In file included from pipy-rs/libs/pipy/src/pjs/pjs.hpp:29,
                 from pipy-rs/libs/pipy/src/api/http.hpp:29,
                 from pipy-rs/libs/pipy/src/admin-proxy.hpp:29,
                 from pipy-rs/libs/pipy/src/admin-proxy.cpp:26:
pipy-rs/libs/pipy/src/pjs/types.hpp:1206:43: error: redefinition of ‘bool __tls_guard’
 1206 | template<class T> thread_local Ref<Class> ClassDef<T>::m_c;
      |                                           ^~~~~~~~~~~
pipy-rs/libs/pipy/src/pjs/types.hpp:1206:43: note: ‘bool __tls_guard’ previously declared here
pipy-rs/libs/pipy/src/pjs/types.hpp:1206: confused by earlier errors, bailing out
make[2]: *** [CMakeFiles/pipy.dir/build.make:90: CMakeFiles/pipy.dir/src/admin-proxy.cpp.o] Error 1
In file included from pipy-rs/libs/pipy/src/pjs/pjs.hpp:29,
                 from pipy-rs/libs/pipy/src/api/algo.hpp:29,
                 from pipy-rs/libs/pipy/src/api/algo.cpp:26:
pipy-rs/libs/pipy/src/pjs/types.hpp:1206:43: error: redefinition of ‘bool __tls_guard’
 1206 | template<class T> thread_local Ref<Class> ClassDef<T>::m_c;
      |                                           ^~~~~~~~~~~
pipy-rs/libs/pipy/src/pjs/types.hpp:1274:50: note: ‘bool __tls_guard’ previously declared here
 1274 | template<class T> thread_local std::vector<Str*> EnumDef<T>::m_val_to_str;
      |                                                  ^~~~~~~~~~
pipy-rs/libs/pipy/src/pjs/types.hpp:1206: confused by earlier errors, bailing out
make[2]: *** [CMakeFiles/pipy.dir/build.make:118: CMakeFiles/pipy.dir/src/api/algo.cpp.o] Error 1
In file included from pipy-rs/libs/pipy/src/pjs/pjs.hpp:29,
                 from pipy-rs/libs/pipy/src/api/http.hpp:29,
                 from pipy-rs/libs/pipy/src/admin-service.hpp:29,
                 from pipy-rs/libs/pipy/src/admin-service.cpp:26:
pipy-rs/libs/pipy/src/pjs/types.hpp:1206:43: error: redefinition of ‘bool __tls_guard’
 1206 | template<class T> thread_local Ref<Class> ClassDef<T>::m_c;
      |                                           ^~~~~~~~~~~
pipy-rs/libs/pipy/src/pjs/types.hpp:1206:43: note: ‘bool __tls_guard’ previously declared here
pipy-rs/libs/pipy/src/pjs/types.hpp:1206: confused by earlier errors, bailing out
make[2]: *** [CMakeFiles/pipy.dir/build.make:104: CMakeFiles/pipy.dir/src/admin-service.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:472: CMakeFiles/pipy.dir/all] Error 2
make: *** [Makefile:136: all] Error 2

system info, test in two machine.

# machine 1:
cmake version 3.29.5
g++ (GCC) 14.1.1 20240522
GNU Make 4.4.1
OS: Arch Linux on Windows 10 x86_64 
Kernel: 5.15.146.1-microsoft-standard-WSL2 

# machine 2:
cmake version 3.18.4
g++ (Debian 10.2.1-6) 10.2.1 20210110
GNU Make 4.3
 OS: Debian GNU/Linux 11 (bullseye) x86_64 
Kernel: 5.10.210-201.855.amzn2.x86_64 

[pajama-coder]

@Hou-Xiaoxuan Currently gcc/g++ isn't supported while compiling pipy. You'll have to install and compile with clang. Please try:

export CC=clang
export CXX=clang

and then cmake and make again

[Hou-Xiaoxuan]

@Hou-Xiaoxuan Currently gcc/g++ isn't supported while compiling pipy. You'll have to install and compile with clang. Please try:

export CC=clang
export CXX=clang

and then cmake and make again

The previous problem was solved, but a new problem was encountered, which seemed to be a problem with zlib's compilation options.

[100%] Building C object CMakeFiles/pipy.dir/deps/sqlite-3.43.2/sqlite3.c.o
[100%] Linking CXX shared library libpipy.so
/usr/bin/ld: deps/zlib-1.3/libz.a(trees.c.o): warning: relocation against `_length_code' in read-only section `.text'
/usr/bin/ld: deps/zlib-1.3/libz.a(trees.c.o): relocation R_X86_64_PC32 against symbol `_length_code' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: bad value
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [CMakeFiles/pipy.dir/build.make:2330: libpipy.so] Error 1
make[1]: *** [CMakeFiles/Makefile2:472: CMakeFiles/pipy.dir/all] Error 2
make: *** [Makefile:136: all] Error 2

[pajama-coder]

@Hou-Xiaoxuan This issue is supposed to be solved in 370bf9aff8650fd7c7e930bbfbfcb4edbf433b36 where zlib version was bumped to v1.3.1 already. Please check with that version and see if it works.

[Hou-Xiaoxuan]

@Hou-Xiaoxuan This issue is supposed to be solved in 370bf9a where zlib version was bumped to v1.3.1 already. Please check with that version and see if it works.

No, updating to the latest source didn't solve my problem. According to the error message, I added set(CMAKE_POSITION_INDEPENDENT_CODE ON) at the top of CMakeLists.txtand compiled successfully.

Could you please merge this change into the main branch, placing it where appropriate (since this option seems to be effective only when compiling dynamic libraries.

For example(I don't familiar with other options):

[naqvis]

No, updating to the latest source didn't solve my problem. According to the error message, I added set(CMAKE_POSITION_INDEPENDENT_CODE ON) at the top of CMakeLists.txtand compiled successfully.

POSITION_INDEPENDENT_CODE or -fPIC flag is turned ON by default for SHARED and MODULE library targets. And this holds true for current CMAKE file. One can verify that this flag is handled properly by cmake via adding -DCMAKE_EXPORT_COMPILE_COMMANDS=ON when invoking cmake.

I gave it a try on Mac M1 using LLVM@17 and compilation works fine.

Please advise if you are encountering issues on any different platform and steps on how can we reproduce that?

[Hou-Xiaoxuan]

No, updating to the latest source didn't solve my problem. According to the error message, I added set(CMAKE_POSITION_INDEPENDENT_CODE ON) at the top of CMakeLists.txtand compiled successfully.

POSITION_INDEPENDENT_CODE or -fPIC flag is turned ON by default for SHARED and MODULE library targets. And this holds true for current CMAKE file. One can verify that this flag is handled properly by cmake via adding -DCMAKE_EXPORT_COMPILE_COMMANDS=ON when invoking cmake.

I gave it a try on Mac M1 using LLVM@17 and compilation works fine.

Please advise if you are encountering issues on any different platform and steps on how can we reproduce that?

I'm developing under both macos m1 and x86 linux, and macos has never had a similar problem.

I have tested on my local machine (archlinux WSL) and cloud ide(Debian GNU/Linux 11), they all triggered the problem.

test steps:

build git:(main) export CXX=clang++
build git:(main) export CC=clang   
build git:(main) cmake -DPIPY_SHARED=ON ..
build git:(main) make -j4

[naqvis]

have tested on my local machine (archlinux WSL) and cloud ide(Debian GNU/Linux 11), they all triggered the problem.

Thanks for the details. Fix for problem has already been merged