Error While uploading file glacier

[IndrajithS]

MT-AWS-Glacier, Copyright 2012-2014 Victor Efimov http://mt-aws.com/ Version 1.120

PID 1634 Started worker PID 1635 Started worker PID 1636 Started worker PID 1637 Started worker ERROR (parent): With current partsize=16777216MiB we will exceed 10000 parts limit for the file "****" (file size 186955223040)

[vsespb]

hi. could you pls provide command line, which caused this error (and config file, if there were additional options)? specially, did you use option --partsize?

[IndrajithS]

Hi, time $GLACIER_PATH/mtglacier upload-file --config $CONFIG_FILE --vault $VAULT --journal $JOURNAL_LOG --dir $DATA_DIR --filename $DATA_DIR/filename

I have not used --partsize flag (upload file size is around 200GB)

Config File: key= secret=

region: eu-west-1, us-east-1 etc

region=

protocol=http (default) or https

protocol=http

[vsespb]

There is a bug in message text.

Message now:

With current partsize=16777216MiB we will exceed 10000 parts limit for the file "********" (file size 186955223040)

Actual message should be:

With current partsize=16777216 bytes we will exceed 10000 parts limit for the file "********" (file size 186955223040)

i.e. partsize is 16777216 bytes, not 16777216MiB

so, with the default partsize (which is 16MiB) ~ 200 000 MiB / 16 MiB = 12500 parts. and per Amazon API limits you should not exceed 10 000 parts.

so to fix the problem for that file you should just specify, say, --partsize=32 option.

[IndrajithS]

Hi,

Thanks for the reply. Now file get uploaded! May i know what is the usage of --partsize??

[vsespb]

it can be grepped in README https://github.com/vsespb/mt-aws-glacier/blob/master/README.md "part" is Amazon api's entity. and partsize is size of part - size chunk of upload, when uploading files.

lower partsize - slower, more overhead. higher partsize - higher chance request to fail and retry. more parts - easier to make concurrent upload.