search

Panxing4game / cosfs

Fsspec filesystem for Tencent Cloud Object Storage (COS)
Apache License 2.0
0 stars 0 forks source link

move this repo to fsspec #1

Open Panxing4game opened 2 years ago

Panxing4game commented 2 years ago

Hi @fsspec,

I am a data engineer at Tencent and we found it's really convenient for our team to directly read/write our cloud object storage (Tencent Cloud COS) with tools like pandas.

Then I created this repo yesterday and already published it to internal mirror, for pandas read/write parquet.

I think it would be even better if we involve more collaboration on this COSFilesystem. So I was wondering if I can transfer this repo to https://github.com/fsspec or just you guys can create a new one for cosfs, so we can make some contribution together.

(until today I found there is no direct message in github, so I created a issue 😄 )

cc @efiop @martindurant @isidentical

martindurant commented 2 years ago

Am I right in thinking that this has not yet been released on pypi/conda?

At a very quick glance at the code, it seems that you are implementing async methods, but none of them have any async calls (i.e., await statements) - should the whole interface actually be sync (subclass from AbstractFileSystem), or is async something you mean to implement in the future?

Ideally there should be a CI system set up and comprehensive test suite; but in the mean time, a message in the readme stating clearly that the whole repo is a work-in-progress.

In principle, it is fine to move to the fsspec org indeed - and you will remain the admin for the repo. We do not guarantee that we will help you with maintenance, though :)

Panxing4game commented 2 years ago

@martindurant

Thanks for you reply! You are right this repo is just WIP, and has been only released in our internal mirror for my team level usage, will mark that in README. For the async part, yea, I was thinking of implementing that part in the future, basically current code was kinda inspired by s3fs.

And sure we will have a CI and test suite, so that there will be a pypi/conda release after that.

My point to move to fsspec org is that I may invite some contributors in & outside my current team to also maintain this repo, instead of repeating themselves with their cosfs. So I think it may be better off in a right org, instead of my personal account.

So no worries about maintenance burden 😄

If all this suits you, I will transfer this repo to fsspec org. cheers.

martindurant commented 2 years ago

Happy to have it, so long as the readme has the warning. Note that there's no rush - if you direct people to here for now, but then more the repo, ass issues and PRs will also transfer, and github will automatically redirect everything.

In order to move, I think you first need to send the repo to an org you have write access to, and then request a transfer from there. I can accept when I see it.