ahgamut
commented
6 months ago Okay, let's add the libraries first, then we'll figure out how to add the Python extensions. At the start I'll mention that if any of the libraries require cffi then I'd recommend waiting, because we need figure out a better way to handle that first.
Super happy to hear you'd like to add more libraries to the repo! The file config/README.md might help get a general idea of what's going on. For now let's focus specifically on the BUILD.mk posted for libxml2:
# the source will be downloaded from here
LIBXML2_SRC := \
https://gitlab.gnome.org/GNOME/libxml2/-/archive/v2.12.6/libxml2-v2.12.6.tar.gz
# this creates the Makefile code for downloading the source
$(eval $(call DOWNLOAD_SOURCE,lib/libxml2,$(LIBXML2_SRC)))
# for dependencies you would need to do something like
# LIBXML2_DEPS := lib/ncurses lib/readline
# $(eval $(call SPECIFY_DEPS,lib/libxml2,$(LIBXML2_DEPS)))
# these are the args passed to the configure script
# note the $$(COSMOS), that's a Makefile thing
LIBXML2_CONFIG_ARGS = --enable-static --disable-shared\
--prefix=$$(COSMOS) --sysconfdir=/zip/etc --datarootdir=/zip/usr/share\
CFLAGS="-Os"
# this creates the Makefile code for configuring x86_64 and aarch64
$(eval $(call AUTOTOOLS_BUILD,lib/libxml2,$(LIBXML2_CONFIG_ARGS),$(LIBXML2_CONFIG_ARGS)))
# this specifies the script to create fat binaries
# set as $(DUMMYLINK0) if there are no fat binaries to build
o/lib/libxml2/built.fat: FATTEN_COMMAND = $(DUMMYLINK0)the above seems like it should work.
I'm not sure how to start this, I imported the
lib/libxml2/BUILD.mkinlib/BUILD.mkbut I'm not sure why I can't build withmake -j4 libxml2.
You were so close :), the build should start if you try make o/lib/libxml2/built.fat. If you look at the BUILD.mk for datasette you see that make datasette is just a shortcut for make o/python/cpy311-datasette/built.fat.
That ought to get a build started for libxml2. You might need to patch a few source files, but it's better if you don't have to. We'll get to that if necessary. Let's look at the other C libraries you want to build:
LXML requires libxml2 and libxslt. Pillow requires libjpeg, zlib, libtiff, libwebp, maybe openjpeg, other optional ones...
zlib is already included in Cosmo, you can build it via make o/cosmo-repo/base/built.fat. The other libraries are pretty well-known, so I would expect that getting them to build should be ok (less than 10 lines worth of patching for each).
croqaz
Hi!
Thank you for this awesome library and all the time you spent maintaining it! I can tell it's a labor of love.
I would like to embed C libraries inside a Python 3.11 build, and I don't know how. I'm writing a public issue in case someone else might want to do the same thing, so maybe we can learn together.
I wonder, would it be appropriate if I add some C libraries in superconfigure ? I care about LXML and Pillow. LXML requires libxml2 and libxslt. Pillow requires libjpeg, zlib, libtiff, libwebp, maybe openjpeg, other optional ones...
I checked the other libs included in this repo and it looks like we need to download the code from the repo and build it here. I could find the following links:
For the libxml2, I made this file:
I'm not sure how to start this, I imported the
lib/libxml2/BUILD.mkinlib/BUILD.mkbut I'm not sure why I can't build withmake -j4 libxml2.I can make a PR for all of these libraries, if it's acceptable.