Download folder recursively

jhagege commented 6 years ago

Hello, is it possible to download recursively a folder, its files and subdirectories with python sdk ? Thanks !

yami commented 6 years ago

OSS is actually an object storage, so there's no concept of 'folder' or 'directory'. However people usually use '/' in object name to emulate a folder.

For your question, currently there's no out-of-box solution from Python SDK yet. If you just need a tool to achieve the purpose you may

use command line tool ossutil (with ossutil cp -r oss://<your-bucket>/<your-folder> <your-local-folder>). Or
use GUI tool ossbrowser

If you need a python solution, you may use 'ObjectIterator' class, and specify 'prefix' parameter, but without 'delimiter' parameter. By this setup, you can iterate all objects prefixed with 'prefix', that's all objects under a 'folder'.

Or you may specify both 'prefix' and 'delimiter' (setting it to '/'), and then visit each 'sub-folder' by yourself.

For more details, check https://www.alibabacloud.com/help/doc-detail/32032.htm

All above links are in English, you can easily find similar docs in Chinese.

jhagege commented 6 years ago

Thanks, this is very helpful and detailed ! I think we are going to go with the ObjectIterator class.

On Fri, 22 Jun 2018 at 09:09, yami notifications@github.com wrote:

OSS is actually an object storage, so there's no concept of 'folder' or 'directory'. However people usually use '/' in object name to emulate a folder.

For your question, currently there's no out-of-box solution from Python SDK yet. If you just need a tool to achieve the purpose you may

use command line tool ossutil https://www.alibabacloud.com/help/doc-detail/50452.htm (with ossutil cp -r oss:/// ). Or

use GUI tool ossbrowser https://www.alibabacloud.com/help/doc-detail/61872.htm

If you need a python solution, you may use 'ObjectIterator' class, and specify 'prefix' parameter, but without 'delimiter' parameter. By this setup, you can iterate all objects prefixed with 'prefix', that's all objects under a 'folder'.

Or you may specify both 'prefix' and 'delimiter' (setting it to '/'), and then visit each 'sub-folder' by yourself.

For more details, check https://www.alibabacloud.com/help/doc-detail/32032.htm

All above links are in English, you can easily find similar docs in Chinese.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/aliyun/aliyun-oss-python-sdk/issues/117#issuecomment-399330911, or mute the thread https://github.com/notifications/unsubscribe-auth/AHKEDPak0x5Ze2IDhfQbc7kAZE4NVj1mks5t_IoNgaJpZM4UvQ4D .

rhinoceros commented 6 years ago

(not tested)

# -*- coding: utf-8 -*-
import oss2
# 阿里云主账号AccessKey拥有所有API的访问权限，风险很高。强烈建议您创建并使用RAM账号进行API访问或日常运维，请登录 https://ram.console.aliyun.com 创建RAM账号。
auth = oss2.Auth('<yourAccessKeyId>', '<yourAccessKeySecret>')
# Endpoint以杭州为例，其它Region请按实际情况填写。
bucket = oss2.Bucket(auth, 'http://oss-cn-hangzhou.aliyuncs.com', '<yourBucketName>')
# 设置Delimiter参数为正斜线（/）。
for obj in oss2.ObjectIterator(bucket, delimiter = '/'):
    # 通过is_prefix方法判断obj是否为文件夹。
    if obj.is_prefix():  # 文件夹
        print('directory: ' + obj.key)
    else:                # 文件
        print('file: ' + obj.key)
        bucket.get_object_to_file(obj.key, '<yourLocalFile>')

list: https://www.alibabacloud.com/help/zh/doc-detail/32032.htm?spm=a2c63.p38356.b99.169.158b4c19Fqu9GE#%E5%88%97%E4%B8%BE%E6%96%87%E4%BB%B6 download: https://www.alibabacloud.com/help/zh/doc-detail/32031.htm?spm=a2c63.p38356.b99.168.48357d86N2fxqa

aliyun / aliyun-oss-python-sdk

Download folder recursively #117