How do you see S3 meta tags

[wifilocator]

Hi, Is there a way to see HTTP header, as it would be good to be able to see these within the UI, even if dump out logs section ? or is there a log file I can see from the cmdline ? e.g.

x-amz-content-sha256: X-Amz-Date: x-amz-storage-class: x-amz-id-2: x-amz-request-id:

reason, I'm asking I'm want the tester to be able to check the storage-class is set correctly and working. or how are others doing this?

Thanks Mark.

[jakobvogel]

Hello Mark, as I am currently on vacation, without access to a computer, I can only provide some thoughts at this time. If necessary, we can discuss in more detail next week.

I assume we are only talking about the headers sent from your software to S3 Ninja, right?

Obviously, in any case, you could try to sniff the packages, and read the values directly from the network traffic.

S3 Ninja itself stores metadata in files prefixed with a $ character. You could navigate to the folder used by the application (possibly within a Docker container) and read that meta-file. (However, I am not sure whether all your values are retained there at all.)

Within the UI, to my knowledge, there is currently no way to view meta-data. Of course, it would be possible to extend the UI to view the contents of the meta-file mentioned above. (Feel free to provide a pull request! 😉)

So, to sum up, in order to make this all convenient, we probably would need to make sure first that all relevant fields are stored at all, and then provide a new view in the UI.

[wifilocator]

Hi Jakob, thanks for the reply and hope you having a good holiday - I found the $ files, but they only seem to have a few of the headers?

#

Wed Aug 18 16:47:14 CEST 2021

ETag=80dead6e8229ed7b5a306f005ff5f8c5 Content-Type=application/octet-stream

I suggested UI as thought might be good for testing, but obviously, the $ files are good enough. but maybe a URL link off the UI could be a good compromise and keep the UI clean?

[jakobvogel]

Sounds good. However, in order to access the metafile via the UI, we may also need to provide additional routing, I think. That means, it may be an option to properly integrate it into the UI as well. I'll have a look at the code once I am back. (However, I only have limited time available for working on this project. That is why I often need to ask for pull requests instead.)

[wifilocator]

I had a look at your code as it seems there are rules on which properties you are interested in ? - as I would go with just anything starts with "x-amz" or was this a problem for others ?

(in S3Dispatcher.java )

private Map<String, String> parseUploadProperties(WebContext webContext) {
    Map<String, String> properties = Maps.newTreeMap();
    for (String name : webContext.getRequest().headers().names()) {
        String nameLower = name.toLowerCase();
        if (nameLower.startsWith("x-amz-meta-") || "content-md5".equals(nameLower) || "content-type".equals(
                nameLower) || "x-amz-acl".equals(nameLower)) {
            properties.put(name, webContext.getHeader(name));
        }
    }
    return properties;
}

as we are looking to make sure the testers see the "x-amz-storage-class" correctly set.

[jakobvogel]

Hmm, tracing back via git blame, this seems to have been part of the initial commit many years ago. If you plan to create a pull request, just modify this, and we'll see whether the unit tests still work. I will also ask the original author to have a look in this case. Here and now, I would rather assume that the signature-related x-amz headers were left out to keep the properties file nice and clean 😉