patrickdepinguin
commented
3 years ago I'm currently using this patch to support ZSTD_PROGRAMS_LINK_SHARED in the make context.
From afc368e7247abfe89250def5b7673a9ccb79e0b1 Mon Sep 17 00:00:00 2001
From: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Date: Wed, 5 Aug 2020 15:35:01 +0200
Subject: [PATCH] make: support ZSTD_PROGRAMS_LINK_SHARED to link zstd with
libzstd
zstd allows building via make, cmake, meson, and others, but unfortunately
the features and flags used for these build systems are inconsistent.
The cmake version respects an option 'ZSTD_PROGRAMS_LINK_SHARED' to let the
zstd binary link dynamically with its own libzstd. Without it, the zstd
program uses static linking and thus has a big size.
This option is not provided in the 'make' system. There does exist a target
'zstd-dll' which intends to do exactly this, but it does not work out of the
box due to linking issues. These in turn are caused because the lib makefile
passes '-fvisibility=hidden' to the linker, which hides certain symbols that
the zstd program needs. The cmake-based compilation does not pass this
visibility flag, due to which all symbols are visible.
Unfortunately, there is no 'install' rule that will install zstd-dll and
create the derived programs like zstdless etc.
With the above in mind, when ZSTD_PROGRAMS_LINK_SHARED is set in make
context:
- remove '-fvisibility=hidden' from the lib compilation
- alter the 'zstd' target to not include the lib objects directly but link
dynamically with libzstd.
See also https://github.com/facebook/zstd/issues/2261 which reported these
inconsistencies between make and cmake compilation.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
---
lib/Makefile | 5 ++++-
programs/Makefile | 8 +++++++-
2 files changed, 11 insertions(+), 2 deletions(-)
diff --git a/lib/Makefile b/lib/Makefile
index 7c6dff02..8f9f36a6 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -204,7 +204,10 @@ $(LIBZSTD): $(ZSTD_FILES)
else
LIBZSTD = libzstd.$(SHARED_EXT_VER)
-$(LIBZSTD): LDFLAGS += -shared -fPIC -fvisibility=hidden
+$(LIBZSTD): LDFLAGS += -shared -fPIC
+ifndef ZSTD_PROGRAMS_LINK_SHARED
+$(LIBZSTD): LDFLAGS += -fvisibility=hidden
+endif
$(LIBZSTD): $(ZSTD_FILES)
@echo compiling dynamic library $(LIBVER)
$(Q)$(CC) $(FLAGS) $^ $(LDFLAGS) $(SONAME_FLAGS) -o $@
diff --git a/programs/Makefile b/programs/Makefile
index 418ad4e6..8897dcf9 100644
--- a/programs/Makefile
+++ b/programs/Makefile
@@ -169,10 +169,16 @@ $(ZSTDDECOMP_O): CFLAGS += $(ALIGN_LOOP)
zstd : CPPFLAGS += $(THREAD_CPP) $(ZLIBCPP) $(LZMACPP) $(LZ4CPP)
zstd : LDFLAGS += $(THREAD_LD) $(ZLIBLD) $(LZMALD) $(LZ4LD) $(DEBUGFLAGS_LD)
zstd : CPPFLAGS += -DZSTD_LEGACY_SUPPORT=$(ZSTD_LEGACY_SUPPORT)
+ifdef ZSTD_PROGRAMS_LINK_SHARED
+# link dynamically against own library
+zstd : LDFLAGS += -L$(ZSTDDIR) -lzstd
+else
+zstd : $(ZSTDLIB_FILES)
+endif
ifneq (,$(filter Windows%,$(OS)))
zstd : $(RES_FILE)
endif
-zstd : $(ZSTDLIB_FILES) $(ZSTD_CLI_OBJ)
+zstd : $(ZSTD_CLI_OBJ)
@echo "$(THREAD_MSG)"
@echo "$(ZLIB_MSG)"
@echo "$(LZMA_MSG)"
--
2.26.2
cemeyer
eli-schwartz
Describe the bug The compiler and link commands used depend on the build system used, e.g. make vs cmake (maybe same for the other build systems). This is unexpected and undesired.
For example, for libzstd, the makefile specifies -fvisibility=hidden, whereas the cmake version does not. Similarly, the compilation of the 'programs' have many compiler warning flags which are absent for cmake. On the other hand, the cmake system now allows to link the 'zstd' program dynamically to libzstd via -DZSTD_PROGRAMS_LINK_SHARED=ON, but this feature is not supported in the 'make' system. There does exist a 'zstd-dll' target, which does not work because the visibility of symbols in libzstd is set to hidden.
To Reproduce Steps to reproduce the behavior:
Expected behavior I expect the same output files regardless of the build system
Additional context With the upgrade from zstd 1.4.3 to 1.4.5, I notice yet another big increase in size, +500KB on libzstd. I thus wanted to benefit from ZSTD_PROGRAMS_LINK_SHARED=ON, but zstd as packaged in Buildroot is using 'make', hence I stumbled upon these inconsistencies.