search

kylebgorman / pynini

Read-only mirror of Pynini
http://pynini.opengrm.org
Apache License 2.0
122 stars 27 forks source link

Support for python 3.10 #64

Closed sshankar619 closed 1 year ago

sshankar619 commented 1 year ago

pip install fails for python 3.10 Is python 3.10 supported?

why 3.10 is not supported in docker file https://github.com/kylebgorman/pynini/blob/0696c5087d36272dc41c20f9e6ebe351de5f67e5/third_party/Dockerfile#L35

Can wheel file be provided for python 3.10

kylebgorman commented 1 year ago

I'll test it on next release. It wasn't working out of the box and I didn't feel like debugging at the time.

You can always just use Conda instead: it supports Python 3.7-3.10 on MacOS (x86 and the new ARM family), Windows, and Linux (x86).

In contrast, the Dockerfile just generates an alternative wheel for Linux x86_64.

Finally, you can edit and run the Dockerfile yourself if you absolutely need Python 3.10 + Linux + x86_64 support but can't use Conda.

gayu-thri commented 1 year ago

Do we have any update on this?

I have got a similar requirement of higher Python version support for 3.11.

For this case, I understand we can use Conda, but if it could be provided as a pip package for Python 3.11, that would be great as one of the projects I'm working on requires Pynini and it is a poetry-based project with PyPI configured as the default package source, and the existing dependencies are also mostly pip packages.​

kylebgorman commented 1 year ago

I’ll include this in on the next release, but making a release may take me as much as a few engineer-days, and usually requires me to update the rest of the stack too. I can’t promise I’ll get to this in the next few weeks, I’m sorry, but I likely will by sometime in April.I don’t know if it will work but the Dockerfile I use to generate the manylinux wheels is checked in to this repository. You could try to modify it to support 3.10 and 3.11, and if you report that it works with minimal modification I can put the wheels for 3.10-11 on PyPI without upgrading Pynini itself. No pressure though.On Mar 16, 2023, at 5:13 AM, Gayathri Ethiraj @.***> wrote: Do we have any update on this? I have got a similar requirement of higher Python version support for 3.11. For this case, I understand we can use Conda, but if it could be provided as a pip package for Python 3.11, that would be great as one of the projects I'm working on requires Pynini and it is a poetry-based project with PyPI configured as the default package source, and the existing dependencies are also mostly pip packages.​

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

gayu-thri commented 1 year ago

I tried building wheels using the Dockerfile by commenting the below line. https://github.com/kylebgorman/pynini/blob/0696c5087d36272dc41c20f9e6ebe351de5f67e5/third_party/Dockerfile#L35

image

However, when I try to install the respective generated cp wheel file (Ex: pynini-2.1.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl) for Python 3.10 and 3.11 (for both 2.1.5 and 2.1.4 pynini) in an environment and test working of pynini, it's working fine. The latest version of pynini in pypi also has only the cp wheel file for Python 3.9 and it's working.

If cp wheel files are usable, would it be okay for us to have this support for 3.10 and 3.11 as well? Please let me know otherwise if I am missing something here.

kylebgorman commented 1 year ago

Okay, I will try to generate the manylinux wheel for Pynini 2.1.5 on Python 3.10 and Python 3.11. May need some time to get it done.

I am not really interested in maintaining Pynini 2.1.4, which is quite out of date now.

I think you may be misreading the test output there: at the bottom it says all 146 tests passed. Some tests deliberately raise error messages and catch the resulting exceptions, but that doesn't mean the test failed...

gayu-thri commented 1 year ago

I think you may be misreading the test output there: at the bottom it says all 146 tests passed. Some tests deliberately raise error messages and catch the resulting exceptions, but that doesn't mean the test failed...

Thanks for clarifying. If I understand correctly, those tests are some kind of negative testing and hence raising errors.

Okay, I will try to generate the manylinux wheel for Pynini 2.1.5 on Python 3.10 and Python 3.11. May need some time to get it done.

Sure. That shouldn't be a problem, please try it at your earliest convenience.

kylebgorman commented 1 year ago

I just uploaded the cp310 (CPython 3.10) x86_64 manylinux wheel for Pynini 2.1.5. I'll support Python 3.10 on the next release too. The dockerfile setup we have now doesn't seem to pull in Python 3.11 so that may be some additional work in the future.

gayu-thri commented 1 year ago

The dockerfile setup we have now doesn't seem to pull in Python 3.11 so that may be some additional work in the future.

Just wanted to clarify on this point.

When I was trying to generate wheel files using the Dockerfile, I was able to get Python 3.11 pulled (cp311) and the above message that I shared earlier stating that it's working fine in Python 3.11 also was by making use of this wheel file.

image

kylebgorman commented 1 year ago

I don’t think I saw Python 3.11 wheels on my system but it’s possible I missed something. Is it likely we got different outputs from the same dockerfile?

On Wed, Apr 5, 2023 at 1:34 AM Gayathri Ethiraj @.***> wrote:

Just wanted to follow up on this, it would be helpful if you could provide me with clarification on this matter

@kylebgorman https://github.com/kylebgorman

The dockerfile setup we have now doesn't seem to pull in Python 3.11 so that may be some additional work in the future.

Just wanted to clarify on this point.

When I was trying to generate wheel files using the Dockerfile, I was able to get Python 3.11 pulled (cp311) and the above message that I shared earlier stating that it's working fine in Python 3.11 also was by making use of this wheel file.

[image: image] https://user-images.githubusercontent.com/39553592/229047449-674b7603-1d90-4757-8a8a-23622c850f9e.png

— Reply to this email directly, view it on GitHub https://github.com/kylebgorman/pynini/issues/64#issuecomment-1497051273, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABG4OOKO7QZUBO7T5LNPLLW7UOCBANCNFSM6AAAAAAUSU45UQ . You are receiving this because you were mentioned.Message ID: @.***>

gayu-thri commented 1 year ago

I have opened a new issue #65 and replied there (https://github.com/kylebgorman/pynini/issues/65#issuecomment-1498686404), we could have the further discussion there as this issue is already closed and is pertaining to Python 3.10 support.