qsctl copy get wrong object content

[wangzw]

I copy 12254 small files from a bucket to local filesystem. The total data is 534.05 MB.

qsctl cp --no-progress -r qs://xxx/ /local/path/

The command exit with exit code 0.

I got wrong content for some files, which is not consistent with the object content in qingstor bucket.

[root@i-ds1xio2d ~]# python --version
Python 2.7.5
[root@i-ds1xio2d ~]# qsctl -v
qsctl 1.7.2
[root@i-ds1xio2d ~]# uname -a
Linux i-ds1xio2d 3.10.0-693.2.2.el7.x86_64 #1 SMP Tue Sep 12 22:26:13 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
[root@i-ds1xio2d ~]# cat /etc/redhat-release
CentOS Linux release 7.4.1708 (Core)

[Xuanwo]

I'll have a test on this.

[Xuanwo]

Do your files have illegal file names? And can you upgrade to 1.7.3 to have an another test? And which type of inconsistent, smaller, bigger or randem data ?

[wangzw]

send you bucket id in qingcloud ticket

[wangzw]

The latest version in pypi is 1.7.2

[wangzw]

aws s3 cp --endpoint-url 'https://s3.pek3a.qingstor.com' --recursive qs://xxx/ /local/path/

get correct object content

[Xuanwo]

Yes, I'm sure there is a bug related to multi thread, I will fix it.

qsctl cp --no-progress -r qs://xxx/ /local/path/ --worker 1 is a workaround.

[Xuanwo]

It's a SDK bug: https://github.com/yunify/qingstor-sdk-python/blob/master/qingstor/sdk/service/bucket.py#L813

We will fix & release it tomorrow.

[Xuanwo]

And this will also lead to files missing: https://github.com/yunify/qsctl/blob/master/qingstor/qsctl/commands/base.py#L157-L166 , patch is coming.

[Xuanwo]

@wangzw This release https://github.com/yunify/qsctl/releases/tag/1.7.4 should fix this bug.

Thanks for your report.

[Xuanwo]

Closed for fixed.