Open reginaldl opened 3 years ago
Hi Reginald, Thanks for the suggestion. I think that supporting pip is a great idea to be more user-friendly. I'm checking your pull request now.
Though I'm not sure how to make pip packages, separating the core C++ library and the Python pip package seems reasonable. If we include binary objects in the pip package, the maintainer have to prepare many files for each platforms x86, x64, ppc, arm etc. Now that Tkrzw's binary packages are going to be ready on Fedora and Debian, and hopefully other platforms including MacOS in the future, the installation of the core library becomes easier.
Oh Debian packages are coming up?! (and Ubuntu I assume?). That would be great!
As for maintaining per architecture package, I guess you’d have to do that anyway since it’s CPython module, right? Wouldn’t be safer to bundle the lib and the python module together to avoid dealing with API dependencies? Thank you!
Though I'm not sure how the approval process is, there's an on-going Debian package proposal. https://salsa.debian.org/debian/tkrzw
I have no experience of making PIP packages for native libraries. However, this project for a native DBM library doesn't include binary files but include some source files. https://pypi.org/project/py-mdbm/#files I think it's normal to package C or C++ source code in the pip package.
Hi Mikio,
ah I see what you mean now. You'd prefer to package sources and let the package installer to compile them. In that case I'm not sure there's a benefit to having a pip package (besides not typing make/make install). I just tried py-mdbm, it does attempt to compile its sources and it actually fails on my machine :) (macOS). Alternatively, I believe pip supports both types of package for a given version, I'd need to double check but basically you can ship compiled version of your lib for some platforms (say the main ones for instance, linux x86_64, macos, windows) and ship a source package for all other platforms (which will automatically compile on the host machine). As an example, I posted a wheel package here: https://gitlab.com/reginaldl/tkrzw-python/-/blob/staticlink/dist/tkrzw-0.1-cp38-cp38-linux_x86_64.whl
This is a compiled version of tkrzw-python, statically linked with tkrzw (which wouldn't be necessary with tkrzw
system packages). This is just a zip file you can list files with unzip -l tkrzw-0.1-cp38-cp38-linux_x86_64.whl
if you're curious. Note the platform linux_x86_64
. This is just an example in the context of this discussion.
Eventually, I believe this could be automated on Github Actions (they support the 3 major OSes)
Nice. If we can use an automation platform to server binary packages, it's reasonable to utilize it for user's convenience.
BTW, as we discussed in the pull request, the package name can be either small letters or capitalized. Is the package name used as the name in the command line "pip install xxx"? If so, using small letter seems better. Moreover, the Python guideline recommends us to use small letter for module names and package names. https://www.python.org/dev/peps/pep-0008/#package-and-module-names Currently, the module name "tkrzw" and the package name is "Tkrzw". As I'm not particular to this case issue, I just want to be conforming to the convention of the community.
Nice catch! I've updated the pull request accordingly.
I've not found the library in PyPI. Is there anything blocking this?
@estraier First I am a big fan of your database-software since many years
I read in your FAQ that you would prefer other people to provide binaries. I did a prototype of building a wheel for pypi.org. (I will fully automate this of course)
Definitely 2. is more correct, but if we do 1:
Let me know what you think.
EDIT: I started the automation: https://github.com/ganwell/tkrzw-wheels/actions/runs/3578797497
Thanks for the suggestion.
I'm not knowledgeable of pypi.org so I appreciate that you take initiative to manage the pypi package. To me, either #1 or #2 is OK. If you prefer #2, let's go with it.
I'm not knowledgeable of pypi.org so I appreciate that you take initiative to manage the pypi package. To me, either #1 or #2 is OK. If you prefer #2, let's go with it.
Thanks for the interest. I started the automated build of wheels (python binary packages). I plan to support at least musllinux, "normal linux" and macOS. If there aren't any mayor problems I will also support windows. See also PR #5.
I do the variant 1. and give you full owner access to both the github- and the pypi-project. So you have full access in case something happens and I can't support the builds anymore.
I hope this will increase the user-base of your great database-software. Easy install is important for many users.
EDIT: here is how a build looks https://github.com/ganwell/tkrzw-wheels/actions/runs/3588311518. It's not important just to show you how it is automated.
The github workflows are now building and uploading musllinux_1_2_x86_64 and manylinux_2_27_x86_64 to test.pypi.org.
https://test.pypi.org/project/tkrzw/0.1.28.6/#files
https://github.com/ganwell/tkrzw-wheels/actions/runs/3606283119
It was quite some work to get all the details right. I hope I can add macOS next week.
Hi,
First of all, thank you for tkrzw. I love it!!!
I would like to add proper python packaging for this awesome library. So whenever someone needs
tkrzw-python
, they could simply do:pip install tkrzw
I am wondering how we could ship the original
tkrzw
library (estraier/tkrzw) as part of this python package? (So we wouldn't have to compile+installtkrzw
before installing the python wrapper)I'm sending a first diff that updates setup.py a little bit in order to allow easy packaging. I think there are probably 2 ways to include the original tkrzw:
tkrzw.(so|dylib)
to the python package (although I'd have to look up how to correctly update ldconfig for python). This is doable by addingpackages
andpackages_data
to the setup function.I'd appreciate some input, thank you!