large files

[Elzor]

I have some trouble with reading large files. My config:

{large_object, [
                                {max_chunked_objs,  10000  },
                                {max_len_for_obj,   5242880000   },
                                {chunked_obj_len,   5242880   },
                                {threshold_obj_len, 5767168 }
                               ]},`

For example I wrote 2GB test file: s3cmd put /var/tmp/sample2.file s3://vstorage/test.big s3cmd ls s3://vstorage 2013-09-26 19:20 2147479552 s3://vstorage/test.big Then I try get it and have error: s3cmd get s3://vstorage/test.big ./test.big

s3://vstorage/test.big -> ./test.big  [1 of 1]

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    An unexpected error has occurred.
  Please try reproducing the error using
  the latest s3cmd code from the git master
  branch found at:
    https://github.com/s3tools/s3cmd
  If the error persists, please report the
  following lines (removing any private
  info as necessary) to:
   s3tools-bugs@lists.sourceforge.net
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Invoked as: /opt/s3cmd/s3cmd get s3://vstorage/test.big ./test.bigProblem: KeyError: 'content-length'
S3cmd:   1.5.0-alpha3

Traceback (most recent call last):
  File "/opt/s3cmd/s3cmd", line 2180, in <module>
    main()
  File "/opt/s3cmd/s3cmd", line 2105, in main
    cmd_func(args)
  File "/opt/s3cmd/s3cmd", line 481, in cmd_object_get
    response = s3.object_get(uri, dst_stream, start_position = start_position, extra_label = seq_label)
  File "/opt/s3cmd/S3/S3.py", line 452, in object_get
    response = self.recv_file(request, stream, labels, start_position)
  File "/opt/s3cmd/S3/S3.py", line 911, in recv_file
    size_left = int(response["headers"]["content-length"])
KeyError: 'content-length'

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    An unexpected error has occurred.
  Please try reproducing the error using
  the latest s3cmd code from the git master
  branch found at:
    https://github.com/s3tools/s3cmd
  If the error persists, please report the
  above lines (removing any private
  info as necessary) to:
   s3tools-bugs@lists.sourceforge.net
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

[yosukehara]

LeoFS-Gateway respond an object with chunked transfer encording(not using content-length). Unfortunately, "s3cmd" does not support that, yet.

[Elzor]

Thanks!

[yosukehara]

I've added the note on the documentation: http://www.leofs.org/docs/s3_client.html#connecting-to-leofs-using-s3cmd