search

sagemath / sage

Main repository of SageMath
https://www.sagemath.org
Other
1.4k stars 474 forks source link

upgrade lrcalc to 2.1 #31355

Closed edd8e884-f507-429a-b577-5d554626c0fe closed 2 years ago

edd8e884-f507-429a-b577-5d554626c0fe commented 3 years ago

lrcalc 2.1 is out https://bitbucket.org/asbuch/lrcalc/src/master/ChangeLog

Instead of rewriting Sage's Python bindings to support the new API, we replace them with a wrapper around the new upstream provided bindings

Upstream:

lrcalclib-2.1.tar.gz (lrcalc): https://sites.math.rutgers.edu/~asbuch/lrcalc/lrcalc-2.1.tar.gz lrcalc-2.1.tar.gz (lrcalc_python): https://pypi.io/packages/source/l/lrcalc/lrcalc-2.1.tar.gz

CC: @antonio-rojas @anneschilling @fchapoton @tscrim @thierry-FreeBSD @orlitzky @mkoeppe @kiwifb @asbuch @slel

Component: packages: optional

Keywords: upgrade, lrcalc

Author: Antonio Rojas, Matthias Koeppe

Branch/Commit: de4fe09

Reviewer: Matthias Koeppe, Travis Scrimshaw

Issue created by migration from https://trac.sagemath.org/ticket/31355

slel commented 3 years ago

Changed keywords from none to upgrade, lrcalc

slel commented 3 years ago
comment:46

Cc-ing the lrcalc maintainer (hi Anders!).

asbuch commented 3 years ago
comment:47

I would like to have lrcalc available on PyPI, but I have not had time to figure out how to put it there. I don't know how to make sure the Python bindings can find the C library, but if PyPI has some magic that can handle that with the above command, then great. I tried this:

python3 setup.py sdist
twine upload --repository testpypi dist/*

That appeared to work, and resulted in https://test.pypi.org/project/lrcalc/ appearing on TestPyPI. However, when I use the pip command displayed there, I get a lot of errors. I'm not eager to have a defunct project in public view on PyPI.

All that said, I can't help thinking that if Sage was able to include the previous version of lrcalc that did not come with Python bindings, but is unable to include the new version, with or without using the supplied bindings, then something in the project must have gotten less flexible in the last few years.

mkoeppe commented 3 years ago

Changed branch from u/arojas/upgrade_lrcalc_to_2_0 to u/mkoeppe/upgrade_lrcalc_to_2_0

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Changed commit from 1153526 to 0bb238b

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

0bb238bbuild/pkgs/lrcalc_python/spkg-install.in: Fixup for sdist layout
antonio-rojas commented 3 years ago
comment:50

Replying to @asbuch:

All that said, I can't help thinking that if Sage was able to include the previous version of lrcalc that did not come with Python bindings, but is unable to include the new version, with or without using the supplied bindings, then something in the project must have gotten less flexible in the last few years.

The thing is that the API changes in lrcalc 2 require an almost complete rewrite of sage's interface, which would essentially make it a duplicate of your python bindings, and doing that doesn't make much sense to me.

Another solution would be to include a copy of your pyx in sage, if the licence allows it.

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

fabfb2bbuild/pkgs/lrcalc_python/package-version.txt: 2.0.0
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Changed commit from 0bb238b to fabfb2b

mkoeppe commented 3 years ago
comment:52

Replying to @asbuch:

I don't know how to make sure the Python bindings can find the C library

No need to worry about this. It is the responsibility of the user to install the C library. This is normal.

But it would be a good idea to explain this in the Python package's README (which would appear in the "Project description" at https://test.pypi.org/project/lrcalc/#description)

See https://pypi.org/project/pplpy/ for inspiration.

mkoeppe commented 3 years ago
comment:53

In lrcalc/python, probably best to change README.Python to either README.md or README.rst

mkoeppe commented 3 years ago
comment:54

The file on TestPyPI does not include the .pyx and .pxd files but it ships a generated .c file that is not needed. To fix this, you can create a file MANIFEST.in

asbuch commented 3 years ago
comment:55

Replying to @antonio-rojas:

Replying to @asbuch:

All that said, I can't help thinking that if Sage was able to include the previous version of lrcalc that did not come with Python bindings, but is unable to include the new version, with or without using the supplied bindings, then something in the project must have gotten less flexible in the last few years.

The thing is that the API changes in lrcalc 2 require an almost complete rewrite of sage's interface, which would essentially make it a duplicate of your python bindings, and doing that doesn't make much sense to me.

Another solution would be to include a copy of your pyx in sage, if the licence allows it.

This is what I imagined would happen when I wrote the Python bindings. I think I also suggested it at some point. Lrcalc is GPL v3, is Sage not compatible with that?

mkoeppe commented 3 years ago
comment:56

The Sage library is "GPL v2 or later", I don't think we can put a GPL v3-only module in.

mkoeppe commented 3 years ago
comment:57

If you relicense the Python bindings to GPL v2 or later, people here on the ticket can probably adapt your code and put it in the Sage library; in this case, we don't need to bother you with Python packaging details.

asbuch commented 3 years ago
comment:58

Replying to @mkoeppe:

The file on TestPyPI does not include the .pyx and .pxd files but it ships a generated .c file that is not needed. To fix this, you can create a file MANIFEST.in

Thanks, this is helpful. Do you know a place where I can find a minimal MANIFEST.in to modify?

mkoeppe commented 3 years ago
comment:59

https://gitlab.com/videlec/pplpy/-/blob/master/MANIFEST.in is a good example

asbuch commented 3 years ago
comment:60

Replying to @mkoeppe:

The Sage library is "GPL v2 or later", I don't think we can put a GPL v3-only module in.

You can hereby consider any files you need to copy as also licensed "GPL v2 or later". Is this official enough?

asbuch commented 3 years ago
comment:61

Replying to @mkoeppe:

https://gitlab.com/videlec/pplpy/-/blob/master/MANIFEST.in is a good example

Thanks, I will give that a try then.

mkoeppe commented 3 years ago
comment:62

Replying to @asbuch:

Is this official enough?

Pretty sure it is, thanks

asbuch commented 3 years ago
comment:63

Replying to @asbuch:

Replying to @mkoeppe:

The file on TestPyPI does not include the .pyx and .pxd files but it ships a generated .c file that is not needed. To fix this, you can create a file MANIFEST.in

Thanks, this is helpful. Do you know a place where I can find a minimal MANIFEST.in to modify?

Now the .pyx and .pxd files are included and the .c file is gone, but I get the same result when I try to install with pip. I have liblrcalc installed in /usr/local/lib.

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

642747abuild/pkgs/lrcalc_python: Use 2.0.1 from TestPyPI
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Changed commit from fabfb2b to 642747a

mkoeppe commented 3 years ago
comment:65

Works fine now here on the branch

mkoeppe commented 3 years ago
comment:66

Replying to @asbuch:

I get the same result when I try to install with pip. I have liblrcalc installed in /usr/local/lib.

If you post some logs, I can take a look

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

81863cfbuild/pkgs/lrcalc_python/dependencies: Add cython
7b76791build/pkgs/lrcalc_python/install-requires.txt: New
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Changed commit from 642747a to 7b76791

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Changed commit from 7b76791 to 482bf57

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

482bf57build/pkgs/lrcalc_python/SPKG.rst: Fix title
mkoeppe commented 3 years ago

Work Issues: Switch to pypi.io URL when released

asbuch commented 3 years ago
comment:70

Replying to @mkoeppe:

Replying to @asbuch:

I get the same result when I try to install with pip. I have liblrcalc installed in /usr/local/lib.

If you post some logs, I can take a look

Thanks! I created a new Python environment with "mkvirtualenv lrcalc_test", then I tried to install with pip and got the following result:

> pip install -i https://test.pypi.org/simple/ lrcalc
Looking in indexes: https://test.pypi.org/simple/
Collecting lrcalc
  Downloading https://test-files.pythonhosted.org/packages/d4/c8/c3988b0714fd225cdc4664f79b57161b8292c43eaad3ff81698d3db58147/lrcalc-2.0.1.tar.gz (15 kB)
  Installing build dependencies ... error
  ERROR: Command errored out with exit status 1:
   command: /home/user/lib/virtualenvs/lrcalc_test/bin/python /home/user/lib/virtualenvs/lrcalc_test/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-cz4e5gzo/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://test.pypi.org/simple/ -- setuptools cython wheel
       cwd: None
  Complete output (3 lines):
  Looking in indexes: https://test.pypi.org/simple/
  ERROR: Could not find a version that satisfies the requirement setuptools (from versions: none)
  ERROR: No matching distribution found for setuptools
  ----------------------------------------
ERROR: Command errored out with exit status 1: /home/user/lib/virtualenvs/lrcalc_test/bin/python /home/user/lib/virtualenvs/lrcalc_test/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-cz4e5gzo/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://test.pypi.org/simple/ -- setuptools cython wheel Check the logs for full command output.
mkoeppe commented 3 years ago
comment:71

When you use -i, it overrides the default package index URL, so it cannot find setuptools any more. (It does not matter that setuptools is already installed in your venv -- this is Python's build isolation.)

mkoeppe commented 3 years ago
comment:72

You can use --extra-index-url instead of -i.

asbuch commented 3 years ago
comment:73

Replying to @mkoeppe:

You can use --extra-index-url instead of -i.

That made a difference, thanks! I hope this problem will go away if I post to PyPI? I assume the PyPI website will show the command "pip install lrcalc"?

mkoeppe commented 3 years ago
comment:74

Replying to @asbuch:

I hope this problem will go away if I post to PyPI? I assume the PyPI website will show the command "pip install lrcalc"?

That's right.

asbuch commented 3 years ago
comment:75

Here it is: https://pypi.org/project/lrcalc/

Many thanks to Matthias for all the help with this!

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Changed commit from 482bf57 to 8ff6dea

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

b4f8b03build/pkgs/lrcalc_python: Use version 2.1 released on PyPI
9286b2dbuild/pkgs/lrcalc/checksums.ini: Rename tarball to disambiguate
8ff6deabuild/pkgs/lrcalc_python/package-version.txt: Update
mkoeppe commented 3 years ago
comment:77

Replying to @asbuch:

Here it is: https://pypi.org/project/lrcalc/

Looking great!

mkoeppe commented 3 years ago

Changed work issues from Switch to pypi.io URL when released to none

mkoeppe commented 3 years ago
comment:79

There's one more detail in the actual C library lrcalc. In the lib..._la_LDFLAGS in Makefile.am, it would be good to add the flag -no-undefined - similar to this change: https://github.com/dimpase/autocliquer/commit/ea653f1f65fc0cdd656f7be45726f844862f2229

This will enable building a shared library on the Cygwin platform (see #29152, #30814).

mkoeppe commented 3 years ago
comment:80

Also the tarballs for the library and the Python sdist have the same name. I have already worked around this on the Sage side by having the library's tarball renamed, so no action is needed. But it might be a good idea to avoid this potential confusion.

mkoeppe commented 3 years ago

Reviewer: Matthias Koeppe

asbuch commented 3 years ago
comment:82

Replying to @mkoeppe:

Also the tarballs for the library and the Python sdist have the same name. I have already worked around this on the Sage side by having the library's tarball renamed, so no action is needed. But it might be a good idea to avoid this potential confusion.

Thanks for the comments! I added the -no-undefined flag. I may try to change the PyPI tarball name to lrcalc-bindings-x.y.z.tar.gz next time I need to make changes. (Maybe there is an option to setuptools to make this happen automagically?)

mkoeppe commented 3 years ago
comment:83

I am not aware of an option to customize it in setuptools, I think it's a fixed format that uses the distribution package's name (lrcalc). Maybe easier to rename the C library package in configure.ac

asbuch commented 3 years ago
comment:84

Replying to @mkoeppe:

I am not aware of an option to customize it in setuptools, I think it's a fixed format that uses the distribution package's name (lrcalc). Maybe easier to rename the C library package in configure.ac

Ok, thanks. In that case I will see if it works if I change the name of the .tar.gz before I upload it with twine. I don't want to change the name I have used for the main package for many years. If the name conflict is a problem, then I would rather change the name of the PyPI project.

edd8e884-f507-429a-b577-5d554626c0fe commented 3 years ago

Changed author from Thierry Monteil, Antonio Rojas to Antonio Rojas

tscrim commented 3 years ago
comment:86

Needs a rebase :/

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

7a0bf0aMerge tag '9.5.beta2' into t/31355/upgrade_lrcalc_to_2_0
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Changed commit from 8ff6dea to 7a0bf0a