liormizr / s3path

s3path is a pathlib extension for AWS S3 Service
Apache License 2.0
208 stars 39 forks source link

`s3path` broken in Python 3.12 due to `_PosixFlavor` no longer being available #149

Closed iamthebot closed 9 months ago

iamthebot commented 12 months ago

Relevant line.

_PosixFlavor is no longer in PathLib in Python 3.12. You probably want PurePosixPath for Python >=3.12 ?

matthewgdv commented 11 months ago

Just to add to this, pathlib._is_wildcard_pattern has also been removed, so that also needs to be updated. I've monkey-patched s3path to work on 3.12 by adding these lines before doing an s3path import, but it would be nice if this could be fixed properly to add 3.12 compatibility.

import pathlib
pathlib._PosixFlavour = pathlib.PurePosixPath
pathlib._is_wildcard_pattern = lambda pat: "*" in pat or "?" in pat or "[" in pat
from s3path import S3Path

EDIT: Actually, this does not really fix anything. The interface has completely changed. PurePath._flavour is no longer an object in 3.12, but seems to now be a module. A larger refactor job will likely be necessary to upgrade.

iamthebot commented 10 months ago

@liormizr any objections if I take a stab at this refactor in a PR? Probably won't get around to it until early jan

We (airbnb) are using this package pretty heavily in some libraries and upgrading anything to Py 3.12 is blocked on this. Would rather coordinate w/ you if you had any opinions.

schultzm commented 10 months ago

+1. If no updates are planned, I'm going to have to abandon my use of s3path from here on.

liormizr commented 10 months ago

Hi guys,

As many of you know I'm living in Israel so as you can guess this project is not on top of my priorities at the moment.

@iamthebot I'l start checking when will it take to do the change to add Python 3.12 support In any case I'll be happy for help and PR's from you guys. :-)

@schultzm do what is best for you.

liormizr commented 10 months ago

On Python Version 3.13 we are expecting a refactor in pathlib with the edition of the new pathlib.VirtualPath class You can see the discussion in issue #140

I added Python 3.12 support in mind of the coming python versions and how s3path will support multiple versions with braking changes in the future. Not sure that it will stay for the long run, but for now it will help us deal quickly without changes for the older versions.

I'll update when we will deploy new version, meanwhile you all are welcome to review and tell me your feedback.

iamthebot commented 10 months ago

As many of you know I'm living in Israel so as you can guess this project is not on top of my priorities at the moment.

Totally understand @liormizr. I hope you're able to stay safe. Totally willing to lend a hand where I can. We use this pretty heavily and it's an awesome project. But your own safety and wellbeing come first of course.

I added Python 3.12 support in mind of the coming python versions and how s3path will support multiple versions with braking changes in the future.

Thanks, will review this PR tomorrow and do some testing with it against our codebase. At a quick glance, looks like this should do the trick.

On Python Version 3.13 we are expecting a refactor in pathlib with the edition of the new pathlib.VirtualPath class

You can see the discussion in issue https://github.com/liormizr/s3path/issues/140 Luckily we reasonable have some time until that lands.

iamthebot commented 10 months ago

OK upon review looks like the 3.12 PR should suffice in closing this issue when merged.

liormizr commented 9 months ago

New Version deployed 0.5.1 with this card fix