php-opencloud / openstack

PHP SDK for OpenStack clouds
Apache License 2.0
222 stars 149 forks source link

ObjectStore: manifest not present in object created via createLargeObject() #328

Open olivbd opened 3 years ago

olivbd commented 3 years ago

I followed the sample code samples/objectstore/v1/objects/download.php.

<?php
require 'vendor/autoload.php';

$openstack = new OpenStack\OpenStack([
    'authUrl' => '{authUrl}',
    'region'  => '{region}',
    'user'    => [
        'id'       => '{userId}',
        'password' => '{password}'
    ],
    'scope'   => ['project' => ['id' => '{projectId}']]
]);

/** @var \GuzzleHttp\Stream\Stream $stream */
$stream = $openstack->objectStoreV1()
                    ->getContainer('{containerName}')
                    ->getObject('{objectName}')
                    ->download();

file_put_contents('my_dest_file.bin', $stream->getContents());

It's working for single objects created with createObject() but no with createLargeObject() ! The returned stream is empty...

Please could you provide a sample code to download large files?

olivbd commented 3 years ago

My bad! The code for download a single object should be same as for download large object!

So, the problem concerns only uploading large object. It seems that the manifest isn't present in the object stat. I will investigate this.

olivbd commented 3 years ago

I uploaded a 50MiB file with the createLargeObject() method and I confirm there is no manifest header!

$service = $openstack->objectStoreV1();
$options = [
    'name' => 'my_large_file',
    'stream' => new Stream(fopen('my_large_file', 'r')),
    'segmentSize' => 10000000,
    'segmentContainer' => 'my_container_segments'
];

$service
    ->getContainer('my_container')
    ->createLargeObject($options)
        ->mergeMetadata(["meta1" => "value1"]);
swift stat my_container my_large_file
               Account: AUTH_*********
             Container: my_container
                Object: my_large_file
          Content Type: application/octet-stream
        Content Length: 0
         Last Modified: Thu, 11 Mar 2021 18:41:32 GMT
                  ETag: d41d8cd98f00b204e9800998ecf8427e
            Meta Meta1: value1
         Accept-Ranges: bytes
           X-Timestamp: 1615488091.51685
            X-Trans-Id: tx***-00604b25c3
X-Openstack-Request-Id: tx***-00604b25c3

If I upload the same file with swift command line utility:

swift upload -S 10000000 -m "meta1:value1" my_container my_large_file
swift stat my_container my_large_file
               Account: AUTH_*********
             Container: my_container
                Object: my_large_file
          Content Type: application/octet-stream
        Content Length: 52428800
         Last Modified: Fri, 12 Mar 2021 08:30:17 GMT
                  ETag: "169f9812ae243ad6f224cdfdfb0ca460"
              Manifest: my_container_segments/my_large_file/1615316870.353349/52428800/10000000/
            Meta Mtime: 1615316870.353349
            Meta Meta1: value1
         Accept-Ranges: bytes
           X-Timestamp: 1615537816.67280
            X-Trans-Id: tx***-00604b26ad
X-Openstack-Request-Id: tx***-00604b26ad

We can see that:

It explains why I'm unable to download it.