tosfs

TOSFS builds on Volcengine TOS Python SDK to provide a convenient Python filesystem interface for TOS（Tinder Object Storage）.

Installation(TODO)

You can install tosfs via pip from PyPI:

$ pip install tosfs

Quick Start

Init FileSystem

Init via ak/sk

from tosfs.core import TosFileSystem
from tos import StaticCredentialsProvider

tosfs = TosFileSystem(
    key='ak',
    secret='sk',
    endpoint_url='http://tos-cn-beijing.volces.com',
    region='cn-beijing',
    credentials_provider=StaticCredentialsProvider, # optional
)

Init via system env

make sure these envs take effect:

export TOS_ACCESS_KEY=' your ak here '
export TOS_SECRET_KEY=' your sk here '
export TOS_ENDPOINT='http://tos-cn-beijing.volces.com'
export TOS_REGION='cn-beijing'

then init TosFileSystem by setting credentials_provider to EnvCredentialsProvider

import os
from tosfs.core import TosFileSystem
from tos import EnvCredentialsProvider

tosfs = TosFileSystem(
    endpoint_url=os.environ.get("TOS_ENDPOINT"),
    region=os.environ.get("TOS_REGION"),
    credentials_provider=EnvCredentialsProvider, # must
)

Access FS and file operation APIs

After getting the instance of TosFileSystem by following the above guidance, now we can access fs and file operation apis, e.g.

# list
tosfs.ls("")
tosfs.ls("mybucket", detail=False)
tosfs.ls("mybucket/rootpath/", detail=False)

# file read/write
with tosfs.open('bucket/root/text.txt', mode='wb') as f:
    f.write('hello tosfs!')

with tosfs.open('bucket/root/text.txt', mode='rb') as f:
    content = f.read()
    print(content)

Contributing

Contributions are very welcome. To learn more, see the Contributor Guide.

License

Distributed under the terms of the Apache 2.0 license, Tosfs is free and open source software.

Issues

If you encounter any problems, please file an issue along with a detailed description.

volcengine / tosfs

readme