list objects not working?

[starfishmod]

I am using the PHP AWS SDK with no issue except when I perform a list objects:

$response = $this->list_objects($s3bucket,array( 'prefix'=>'u/public/' ));

the logs say localhost - - [31/Aug/2012:11:30:30 EST] "GET /?prefix=u%2F2public%2F HTTP/1.1" 200 300 http://127.0.0.1:4567/?prefix=u%2Fpublic%2F -> /?prefix=u%2Fpublic%2F

and it gives a bucket list as the return.

is this a known issue? am I doing something wrong? The same query works on s3 directly.

Thanks Andrew

[jubos]

Andrew,

Thanks for the bug report. I think there needs to be a PHP AWS SDK unit test file, so we can get to the bottom of it. All the different language SDKs tend to tickle the AWS API calls a little bit differently, so it is all about getting all the edge cases.

[philippe-tseyen]

I am hitting the same issue.

   <dependency>
        <groupId>com.amazonaws</groupId>
        <artifactId>aws-java-sdk</artifactId>
        <version>1.4.3</version>
    </dependency>

You can find my testcase over here:

https://gist.github.com/anonymous/5579687

When I execute this TestCase I get the following xml back from the server: https://gist.github.com/philippe-tseyen/5579744

[petermilan]

I have the same issue, listObjects returns empty structure (AWS-SDK for JavaScript).

[bbCharles]

Yeah listObjects is returning an empty structure for me as well

[fireice009]

Using boto3(1.4.1), and running fake-s3 in https://hub.docker.com/r/lphoward/fake-s3/. Most operations such as head_object, get_object, put_object are ok. But calling client.list_objects throws an exception.

  File "/usr/local/lib/python3.4/dist-packages/botocore/client.py", line 159, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.4/dist-packages/botocore/client.py", line 494, in _make_api_call
    raise ClientError(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (NoSuchKey) when calling the ListObjects operation: The specified key does not exist

On the other hand, If I put an object bar to bucket foo through endpoint_url='http://ct-s3:4569', I get path /fakes3_root/ct-s3/foo/bar in docker. And executingaws --endpoint-url='http://localhost:4569' s3 ls returns ct-s3, not foo. Calling client.list_objects(Bucket='ct-s3')get the same exception.

[tobowers]

having same problem with curl:

http://192.168.99.100:4569/test?list-type=2&max-keys=10&prefix=%2Fownership&start-after=

returning

<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <Name>test</Name>
    <Prefix>/ownership</Prefix>
    <Marker/>
    <MaxKeys>10</MaxKeys>
    <IsTruncated>false</IsTruncated>
</ListBucketResult>

instead of objects

[tobowers]

that URL was generated from the golang sdk

[yjhuoh]

can confirm this doesn't work in node using aws-sdk either.

[CpuID]

having same problem with curl:

http://192.168.99.100:4569/test?list-type=2&max-keys=10&prefix=%2Fownership&start-after=

That example uses ListObjectV2 which I believe may not be supported yet in this project?

[alexmojaki]

While trying to reproduce this issue, I found that it depends on the HTTP Host header:

$ curl -H 'Host: localhost:4569' http://localhost:4569/my_bucket/ 
<?xml version="1.0" encoding="UTF-8"?><ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Name>my_bucket</Name><Prefix/><Marker/><MaxKeys>1000</MaxKeys><IsTruncated>false</IsTruncated><Contents><Key>the_key</Key><LastModified>2017-11-20T14:35:30.139Z</LastModified><ETag>"3a37d03e20b3b3245b460349de1e4057"</ETag><Size>9</Size><StorageClass>STANDARD</StorageClass><Owner><ID>abc</ID><DisplayName>You</DisplayName></Owner></Contents></ListBucketResult>
$ curl -H 'Host: blablabla:4569' http://localhost:4569/my_bucket/
<?xml version="1.0" encoding="UTF-8"?><Error><Code>NoSuchKey</Code><Message>The specified key does not exist</Message><Key>my_bucket</Key><RequestId>1</RequestId><HostId>2</HostId></Error>

The problem seems to be here: https://github.com/jubos/fake-s3/blame/8dc0734c3d6b308dc033541c57f63729e806696a/lib/fakes3/server.rb#L486-L489

[alexmojaki]

OK, I was able to fix the issue by using the -H argument of fakes3 to set the hostname of the server to the same hostname I was using in my client. It should really be documented somewhere that this is necessary. I'm not sure why but my programs seem to continue to work fine without the hostname until I try to list objects.