Issue with swift, please help

[sabyasg]

Hi,

I'm using the swift-client and swift-model part of the SDK and am stuck as container show() is not working as desired. Here is our code snippet at the caller:

container = swift.containers().show(driveName).execute();

However, looking at ContainersResource.java, we found:

            public Show(String containerName) {

// return target.path(containerName).request(MediaType.APPLICATION_JSON).head(); }

Changing this to something similar as ServersResource.java gets me into an exception as below:

    public class Show extends OpenStackRequest<Container> {

            private String containerName;

            public Show(String containerName) {
                    super(CLIENT, HttpMethod.GET, new StringBuilder("/").append(containerName), null, Container.class);
            }

    }

However, I get the exception: com.sun.jersey.api.client.ClientHandlerException: org.codehaus.jackson.map.JsonMappingException: Can not deserialize instance of com.woorea.openstack.swift.model.Containers out of START_ARRAY token

Enabling logs in catalina.out, I found that we are receiving the response as a JSON array for swift: [{"count": 1, "bytes": 3968, "name": "mdic-cache-1"}]

Note, this is an array, when things are something like what Nova returns, the ServersResource works fine I guess..

{"servers": [{"status": "ERROR", ....... }]}

Is a fix available so that I can use swift from the openstack SDK? Please help, I'm stuck..

[woorea]

Hi,

I need to review this code cause it not working as expected

A hint:

public class Show extends OpenStackRequest<List> {

        private String containerName;

        public Show(String containerName) {
                super(CLIENT, HttpMethod.GET, new StringBuilder("/").append(containerName), null, new GenericType<List<Container>>());
        }

}

I have no tested this code but this is the way to solve ... probably you would need to extend OpenStackRequest to support GenericType response

look changes at List in OpenStackRequest and the new GenericType<List>()

let me know if i can help, I will fix it in the next release. If you get it working before please pull request :)

[sabyasg]

Hi Luis,

Many thanks for responding. I'm wondering what needs to be done in OpenStackRequest. Also, what needs to be added to pom.xml for GenericType? Would you be able to provide me with a patch or a set of new files that I can try out on top of 3.2.2?

Best regards, Sabya

On Tuesday, October 29, 2013 6:56 AM, Luis Gervaso notifications@github.com wrote:

Hi,

I need to review this code cause it not working as expected A hint: public class Show extends OpenStackRequest> { private String containerName; public Show(String containerName) { super(CLIENT, HttpMethod.GET, new StringBuilder("/").append(containerName), null, new GenericType<List>()); } } I have no tested this code but this is the way to solve ... probably you would need to extend OpenStackRequest to support GenericType response look changes at List in OpenStackRequest and the new GenericType>() let me know if i can help, I will fix it in the next release. If you get it working before please pull request :) — Reply to this email directly or view it on GitHub.

[woorea]

Hi,

The issue is OpenStackRequest does not have dependencies in jax-rs

I'll review and create a patch asap

[sabyasg]

Hi Luis,

Thanks for the help. Would you please let me know when a patch may be available?

Thanks, Sabya

On Tuesday, October 29, 2013 7:33 AM, Luis Gervaso notifications@github.com wrote:

Hi,

The issue is OpenStackRequest does not have dependencies in jax-rs I'll review and create a patch asap — Reply to this email directly or view it on GitHub.

[sabyasg]

Hi Luis,

Would you please let me know when a patch may be available? I'm totally blocked and cannot progress without this change. If you can send me a patch for this, I can test and make some progress with my work..

Best regards, and Thanks again, Sabya

On Friday, November 1, 2013 7:44 AM, Sabyasachi Sengupta sabyasg@yahoo.com wrote:

Hi Luis,

Thanks for the help. Would you please let me know when a patch may be available?

Thanks, Sabya

On Tuesday, October 29, 2013 7:33 AM, Luis Gervaso notifications@github.com wrote:

Hi,

The issue is OpenStackRequest does not have dependencies in jax-rs I'll review and create a patch asap — Reply to this email directly or view it on GitHub.

[thatsdone]

Hi sabyasg,

Which do you need 'swift stat [container [object]]' equivalent or 'swift list [container]' equivalent? The former corresponds to HEAD and the latter does GET HTTP method.

I wanted to use both and I looked into this issue somewhat. If I understand correctly, I think that there is no way other than creating a stub class (not extending OpenStackRequest) that takes either of (swift) Object or Container class as a Generics parameter and returns List of Object or List of Container. This comes from compilation time behavior of Java Generics.

One more thing I want to point out is that Swift resonses consist of top level JSON Array (without name), not a JSON Object. This makes the situation a bit comlicated. In this sense, ceilometer has the same kind of issue.

I found a workaround, and you can find my POC code below:

https://github.com/thatsdone/openstack-java-sdk/commit/405b7227fc8dc717cc445a1838c9d8701d661e8b

Using above, you can get a list of containers ('swift list' equivalent) by writing like the following.

 List<Container> containers = swiftClient.containers().list().queryParam("format", "json").execute();

I know that the above work around is VERY ugly. But anyway, it's working for me.

If my POC code makes sense for Luis, I will submit a pull request.

Thanks in advance, Masanori

[sabyasg]

Hi Masanori,

Thank you. I will try it. Probably a couple of things to be looked at:

1. How to modify all other entry points like "create", "show", "delete" in ContainersResource to use OpenStackRequestList as all responses from swift will return array type JSON response, right?
2. Minor nit: 'execute' method in OpenStackRequestList should have 'obj' instead of 'swiftObj'.

Can you please send me a new patch for #1? I think it makes sense to have this and modify Ceilometer as well as a workaround. Probably Luis will fix it in the way he wants in his next release.

Thanks, Sabya

On Wednesday, November 13, 2013 9:22 AM, Masanori ITOH notifications@github.com wrote:

Hi sabyasg,

Which do you need 'swift stat [container [object]]' equivalent or 'swift list [container]' equivalent? The former corresponds to HEAD and the latter does GET HTTP method. I wanted to use both and I looked into this issue somewhat. If I understand correctly, I think that there is no way other than creating a stub class (not extending OpenStackRequest) that takes either of (swift) Object or Container class as a Generics parameter and returns List or List. This comes from compilation time behavior of Java Generics. One more thing I want to point out is that Swift resonses consist of top level JSON Array (without name), not a JSON Object. This makes the situation a bit comlicated. In this sense, ceilometer has the same kind of issue. I found a workaround, and you can find my POC code below: thatsdone@405b722 Using above, you can get a list of containers ('swift list' equivalent) by writing like the following. List containers = swiftClient.containers().list().queryParam("format", "json").execute(); I know that the above work around is VERY ugly. But anyway, it's working for me. If my POC code makes sense for Luis, I will submit a pull request. Thanks in advance, Masanori — Reply to this email directly or view it on GitHub.

[thatsdone]

Hi Sabya,

Thanks for the comments.

Thank you. I will try it. Probably a couple of things to be looked at:

1. How to modify all other entry points like "create", "show", "delete" in ContainersResource to use >OpenStackRequestList as all responses from swift will return array type JSON response, right?

Here are some more issues, I think.

All of "stat (show)", "delete", "post", "upload" and "download" do not return a response body, but give us result information via HTTP headers.

What do you think the natural way returning the HTTP header information to callers? Although I have POC codes to the issue which return a String containing all header lines, I think that something like Map<String key, String value> is better. But, that means we need another wrapper class because OpenStackRequestList() returns a List. Here, the problem is, the new wrapper changes the semantics of execute() because it returns a Java Object which does not map JSON response.

I'm wondering if it's acceptable or not.

1. Minor nit: 'execute' method in OpenStackRequestList should have 'obj' instead of 'swiftObj'.

Thanks in advance, Masanori

[sabyasg]

While on the "list" example, I got "java.lang.ClassCastException: java.util.LinkedHashMap cannot be cast to com.woorea.openstack.swift.model.Container" when I try to grab Container out of the List. Here is my client code:

List containers;                 try {                         containers = swift.containers().list().queryParam("format", "json").execute();                 } catch (Exception e) {                         if (verbose) {                                 System.out.println("Exception " + e + " while retrieving list of objects");                         }                         return null;                 }                 for (Container container : containers) {<<<< this line gives the exception

                        String name = container.getName(); [..] rest of my business logic with "name"               }

How are you obtaining individual Container objects?

On Thursday, November 14, 2013 9:06 AM, Masanori ITOH notifications@github.com wrote:

Hi Sabya,

Thanks for the comments. Thank you. I will try it. Probably a couple of things to be looked at:

1. How to modify all other entry points like "create", "show", "delete" in ContainersResource to use >OpenStackRequestList as all responses from swift will return array type JSON response, right? Here are some more issues, I think. All of "stat (show)", "delete", "post", "upload" and "download" do not return a response body, but give us result information via HTTP headers. What do you think the natural way returning the HTTP header information to callers? Although I have POC codes to the issue which return a String containing all header lines, I think that something like Map<String key, String value> is better. But, that means we need another wrapper class because OpenStackRequestList() returns a List. Here, the problem is, the new wrapper changes the semantics of execute() because it returns a Java Object which does not map JSON response. I'm wondering if it's acceptable or not.
   1. Minor nit: 'execute' method in OpenStackRequestList should have 'obj' instead of 'swiftObj'. Thanks in advance, Masanori — Reply to this email directly or view it on GitHub.

[thatsdone]

Hi Sabya,

Humm..

Looks like something is strange.

Here is my small program which I usually use to test my works on openstack-java-sdk: https://github.com/thatsdone/openstack-java-cli

and, the code chunk getting a list of containers is around: https://github.com/thatsdone/openstack-java-cli/blob/master/src/main/java/com/github/thatsdone/jopst/Jswift.java#L91

As you can see, looks like allmost the same with yours.

Indeed that I locally have a further modified version of SDK which returns LinkedHashMap for execute() of show() not of list(). But, I have not pushed that yet, and you use list() instead of show(). This is weird...

Is that possible pushing your SDK code somewhere especially files under swift-client/src letting us see?

Regards, Masanori

[sabyasg]

util.printJson is probably similar to a System.out.println(containers), which also works for me. Did you try getting List to Container objects and then trying to, say, print them as I showed in my last mail? 

On Friday, November 15, 2013 8:11 PM, Masanori ITOH notifications@github.com wrote:

Hi Sabya,

Humm.. Looks like something is strange. Here is my small program which I usually use to test my works on openstack-java-sdk: https://github.com/thatsdone/openstack-java-cli and, the code chunk getting a list of containers is around: https://github.com/thatsdone/openstack-java-cli/blob/master/src/main/java/com/github/thatsdone/jopst/Jswift.java#L91 As you can see, looks like allmost the same with yours. Indeed that I locally have a further modified version of SDK which returns LinkedHashMap for execute() of show() not of list(). But, I have not pushed that yet, and you use list() instead of show(). This is weird... Is that possible pushing your SDK code somewhere especially files under swift-client/src letting us see? Regards, Masanori — Reply to this email directly or view it on GitHub.

[thatsdone]

Hi Sabya,

Here are some examples. Does this make sense for you?

At first, native swift command output.

$ swift list glance mycontainer

$ swift upload mycontainer README.md README.md

$ swift list glance mycontainer

$ swift list mycontainer README.md

Then, my small Java program examples which call my fork of openstack-java-sdk.

The below getting List<Container> hits around:

https://github.com/thatsdone/openstack-java-cli/blob/master/src/main/java/com/github/thatsdone/jopst/Jswift.java#L89

$ bin/jopst swift list [ { "count" : 112, "bytes" : 27662479872, "name" : "glance" }, { "count" : 0, "bytes" : 0, "name" : "mycontainer" } ]

and, the blow getting List<Object> hits around:

https://github.com/thatsdone/openstack-java-cli/blob/master/src/main/java/com/github/thatsdone/jopst/Jswift.java#L100

$ bin/jopst swift list mycontainer [ { "hash" : "40b3d2cf3755d1bb6d13556083cf8ef5", "last_modified" : "2013-11-16T07:05:42.873470", "bytes" : 3356, "name" : "README.md", "content_type" : "application/octet-stream" } ]

Regards, Masanori