liormizr / s3path

s3path is a pathlib extension for AWS S3 Service
Apache License 2.0
206 stars 39 forks source link

Rename to or from FileSystem #107

Closed petrus-v closed 8 months ago

petrus-v commented 1 year ago

Proposal to fix #88 based on ~#106~ #111

liormizr commented 1 year ago

Hi @petrus-v This is a great Idea Do you want to wait until we will finish talking about PR #106 ? Or do you want to split them and look at this as a separate project?

petrus-v commented 1 year ago

Let's decide what we do on #106 first !

Fun fact this code is in production since this night ^^

Still getting some troubles to configure boto3 with scaleway with large files (24Go)

liormizr commented 1 year ago

Hi @petrus-v If you want to fix the issue when doing rename from s3 to local file system or the other way around that's nice And that's the task that I thought you want to do. But like I wrote to you in the URI lib factory PR I don't think that s3path don't need to require uri-lib-facroty I think that it need to be the opposite Maybe this feature need to be in uri-lib-facroty if you want to support all rename options (move from s3 to ftp or filesystem or ....)

petrus-v commented 1 year ago

If you want to fix the issue when doing rename from s3 to local file system or the other way around that's nice And that's the task that I thought you want to do. But like I wrote to you in the URI lib factory PR I don't think that s3path don't need to require uri-lib-facroty I think that it need to be the opposite Maybe this feature need to be in uri-lib-facroty if you want to support all rename options (move from s3 to ftp or filesystem or ....)

indeed this could solve the issue, thanks for feedback.

I haven't deep that much here, this works for the time being and don't expect to be merged as it anyway.

I've another branch that add copy method (ie: my_path.copy(other_path)) which is unfortunate not provided by pathlib interface and allows to copy file from s3 to filesystem that should be some how an equivalent use cases.

I'll think more to takes right decision on #111