search

ContainX / openstack4j

A Fluent OpenStack SDK / Client Library for Java
http://openstack4j.com
Other
290 stars 367 forks source link

blockStorage can't get volumes of project #1067

Open rafael-bertoli opened 7 years ago

rafael-bertoli commented 7 years ago

Hi everyone, when i try to get volumes of project with api V3 the response is "404 Not Found", but if i do request with api V2 all volumes are returned.

See request logs of V2 and V3, i noticed that in V3 the project id is not being sent...

22:40:55.110 [main] DEBUG org.openstack4j.core.transport.internal.HttpExecutor - Executing Request: http://192.168.55.34:5000/v3 -> /auth/tokens jul 19, 2017 10:40:56 PM org.glassfish.jersey.filter.LoggingFilter log INFORMAÇÕES: 1 Sending client request on thread main 1 > POST http://192.168.55.34:5000/v3/auth/tokens 1 > Accept: application/json 1 > Content-Type: application/json 1 > OS4J-Auth-Command: Tokens { "auth" : { "identity" : { "password" : { "user" : { "name" : "qauser", "domain" : { "name" default" }, "password" : "r1q150Sqa" } }, "methods" : [ "password" ] }, "scope" : { project" : { "name" : "qauser", "domain" : { "name" : "default" } } } }} jul 19, 2017 10:40:56 PM org.glassfish.jersey.filter.LoggingFilter log INFORMAÇÕES: 1 Client response received on thread main 1 < 201 1 < Connection: Keep-Alive 1 < Content-Length: 5566 1 < Content-Type: application/json 1 < Date: Wed, 19 Jul 2017 22:41:07 GMT 1 < Keep-Alive: timeout=5, max=100 1 < Server: Apache/2.4.6 (CentOS) mod_wsgi/3.4 Python/2.7.5 1 < Vary: X-Auth-Token 1 < x-openstack-request-id: req-5b48502d-d3cf-4453-a65a-2861cafb9ad2 1 < X-Subject-Token: gAAAAABZb-ADCwa6zyGaOpa0BCKDUJiUTXtXWd7UrH5wZao9fGhhESX5XhgBTFHh4HkEVlpm6DupKQJLRDCLiVvRORlDYCKyXSUmv3tUIGJVd7A0CX3tbqczA6xD007Yy8gbnH8BZkIl0GNG2kRIDtGl3Rk2SvCp0ai29Tvty8-QmCpE0P3rMaM {"token": {"is_domain": false, "methods": ["password"], "roles": [{"id": "cd229e6f1f954b05a9e99eb634ede581", "name": "user"}], "expires_at": "2017-07-19T23:41:07.000000Z", "project": {"domain": {"id": "default", "name": "Default"}, "id": "7685011104884466bd89286f411d518d", "name": "qauser"}, "catalog": [{"endpoints": [{"region_id": "NYQA", "url": "http://glancedev-ny.qa.os.com.br:9292", "region": "NYQA", "interface": "admin", "id": "363d08364c9c467fa52d8d1fc989220f"}, {"region_id": "NYQA", "url": "http://glancedev-ny.qa.os.com.br:9292", "region": "NYQA", "interface": "public", "id": "a4e1ef3c3f2240049f2486a3fd54945e"}, {"region_id": "NYQA", "url": "http://glancedev-ny.qa.os.com.br:9292", "region": "NYQA", "interface": "internal", "id": "d4e26fc923374e799059d9eef5e2e979"}], "type": "image", "id": "0aa8ea973f0c442488746cf2962f933a", "name": "glance"}, {"endpoints": [{"region_id": "NYQA", "url": "http://cinderdev-ny.qa.os.com.br:8776/v3/7685011104884466bd89286f411d518d", "region": "NYQA", "interface": "admin", "id": "a550810eba0d43b280ec1872265e7188"}, {"region_id": "NYQA", "url": "http://cinderdev-ny.qa.os.com.br:8776/v3/7685011104884466bd89286f411d518d", "region": "NYQA", "interface": "internal", "id": "e2a3622bd88f463dac662a6783d60721"}, {"region_id": "NYQA", "url": "http://cinderdev-ny.qa.os.com.br:8776/v3/7685011104884466bd89286f411d518d", "region": "NYQA", "interface": "public", "id": "fa34d9b09ff14ed780e85b2c44d8a2bc"}], "type": "volumev3", "id": "166ca03dde244f0c8825099bd14d11fa", "name": "cinderv3"}, {"endpoints": [{"region_id": "NYQA", "url": "http://novadev-ny.qa.os.com.br:8778", "region": "NYQA", "interface": "admin", "id": "5d45b8edea0447aba49b0d7ea127d0d1"}, {"region_id": "NYQA", "url": "http://novadev-ny.qa.os.com.br:8778", "region": "NYQA", "interface": "public", "id": "8dfb959965af48c5a8cdeaee9562af28"}, {"region_id": "NYQA", "url": "http://novadev-ny.qa.os.com.br:8778", "region": "NYQA", "interface": "internal", "id": "dda0a7283e3748c6b4efa3c793649f73"}], "type": "placement", "id": "403d87686acf4c82949b2d978362ccad", "name": "placement"}, {"endpoints": [{"region_id": "NYQA", "url": "http://cinderdev-ny.qa.os.com.br:8776/v2/7685011104884466bd89286f411d518d", "region": "NYQA", "interface": "admin", "id": "0a0b863cebd2447c880f7e2249b2f0dd"}, {"region_id": "NYQA", "url": "http://cinderdev-ny.qa.os.com.br:8776/v2/7685011104884466bd89286f411d518d", "region": "NYQA", "interface": "public", "id": "c51dff3e65b348ffbf953bbc197d0130"}, {"region_id": "NYQA", "url": "http://cinderdev-ny.qa.os.com.br:8776/v2/7685011104884466bd89286f411d518d", "region": "NYQA", "interface": "internal", "id": "df7c82567a7c489f8a88bbc163480821"}], "type": "volumev2", "id": "43bad8d10a6f4c26a0ae89a9a3170c70", "name": "cinderv2"}, {"endpoints": [{"region_id": "NYQA", "url": "http://controller01dev-zone01-ny.qa.os.com.br:5000/v3/", "region": "NYQA", "interface": "internal", "id": "3e46167a3f0e45a7bb99bcf775811ec7"}, {"region_id": "NYQA", "url": "http://controller01dev-zone01-ny.qa.os.com.br:35357/v3/", "region": "NYQA", "interface": "admin", "id": "8f47e5d355ba48dd96726c84c05f2ea1"}, {"region_id": "NYQA", "url": "http://controller01dev-zone01-ny.qa.os.com.br:5000/v3/", "region": "NYQA", "interface": "public", "id": "b4d9db8e256d48fea83bdafa9199cb51"}], "type": "identity", "id": "6032290fd91c4448a6f26e17470b107c", "name": "keystone"}, {"endpoints": [{"region_id": "NYQA", "url": "http://designatedev-ny.qa.os.com.br:9001/", "region": "NYQA", "interface": "public", "id": "477ec11abcbe42a696fb7f5864e8e4cc"}, {"region_id": "NYQA", "url": "http://designatedev-ny.qa.os.com.br:9001/", "region": "NYQA", "interface": "admin", "id": "4b59ea48796d44499bb85ddfc1611996"}, {"region_id": "NYQA", "url": "http://designatedev-ny.qa.os.com.br:9001/", "region": "NYQA", "interface": "internal", "id": "5c1cca0fa41e45e99801b470c21e9449"}], "type": "dns", "id": "a57f63c435fe471dbc8f057b154dba8e", "name": "designate"}, {"endpoints": [{"region_id": "NYQA", "url": "http://novadev-ny.qa.os.com.br:8774/v2.1", "region": "NYQA", "interface": "public", "id": "7878b32ac4b1460487b9e3000cb8d103"}, {"region_id": "NYQA", "url": "http://novadev-ny.qa.os.com.br:8774/v2.1", "region": "NYQA", "interface": "internal", "id": "dca66bf909164a7687ccf7470c2bb1ab"}, {"region_id": "NYQA", "url": "http://novadev-ny.qa.os.com.br:8774/v2.1", "region": "NYQA", "interface": "admin", "id": "e65fc9a1bbb74f8e9b4adfcb0f541a44"}], "type": "compute", "id": "b28f94c54a824b6898be385b9c873439", "name": "nova"}, {"endpoints": [{"region_id": "NYQA", "url": "http://neutrondev-ny.qa.os.com.br:9696", "region": "NYQA", "interface": "public", "id": "32eb9979290f4878b4a5026a743ffbb9"}, {"region_id": "NYQA", "url": "http://neutrondev-ny.qa.os.com.br:9696", "region": "NYQA", "interface": "internal", "id": "a88016c3a5cd4ce19157979fd7722e51"}, {"region_id": "NYQA", "url": "http://neutrondev-ny.qa.os.com.br:9696", "region": "NYQA", "interface": "admin", "id": "b686de6e54df43438d9d3faebb65c326"}], "type": "network", "id": "b86fc16070a94bf1b73d05e9a401414f", "name": "neutron"}], "user": {"password_expires_at": null, "domain": {"id": "default", "name": "Default"}, "id": "059c54a608f946719ed63e2795a5f152", "name": "qauser"}, "audit_ids": ["nEgXusMaT8aqSIQgDp6bYQ"], "issued_at": "2017-07-19T22:41:07.000000Z"}} 22:40:56.782 [main] DEBUG org.openstack4j.core.transport.internal.HttpExecutor - Executing Request: http://192.168.55.34:5000/v3 -> /volumes/detail jul 19, 2017 10:40:56 PM org.glassfish.jersey.filter.LoggingFilter log INFORMAÇÕES: 1 Sending client request on thread main 1 > GET http://192.168.55.34:5000/v3/volumes/detail 1 > Accept: application/json 1 > User-Agent: OpenStack4j / OpenStack Client 1 > X-Auth-Token: gAAAAABZb-ADCwa6zyGaOpa0BCKDUJiUTXtXWd7UrH5wZao9fGhhESX5XhgBTFHh4HkEVlpm6DupKQJLRDCLiVvRORlDYCKyXSUmv3tUIGJVd7A0CX3tbqczA6xD007Yy8gbnH8BZkIl0GNG2kRIDtGl3Rk2SvCp0ai29Tvty8-QmCpE0P3rMaM jul 19, 2017 10:40:57 PM org.glassfish.jersey.filter.LoggingFilter log INFORMAÇÕES: 1 Client response received on thread main 1 < 404 1 < Connection: Keep-Alive 1 < Content-Length: 93 1 < Content-Type: application/json 1 < Date: Wed, 19 Jul 2017 22:41:07 GMT 1 < Keep-Alive: timeout=5, max=99 1 < Server: Apache/2.4.6 (CentOS) mod_wsgi/3.4 Python/2.7.5 1 < Vary: X-Auth-Token 1 < x-openstack-request-id: req-76f7f4f8-0742-4af0-8309-dc336e57c1a1 {"error": {"message": "The resource could not be found.", "code": 404, "title": "Not Found"}}

V2 log

22:22:38.528 [main] DEBUG org.openstack4j.core.transport.internal.HttpExecutor - Executing Request: http://192.168.55.34:5000/v2.0 -> /tokens jul 19, 2017 10:22:39 PM org.glassfish.jersey.filter.LoggingFilter log INFORMAÇÕES: 1 Sending client request on thread main 1 > POST http://192.168.55.34:5000/v2.0/tokens 1 > Accept: application/json 1 > Content-Type: application/json 1 > OS4J-Auth-Command: Tokens { "auth" : { "tenantName" : "qauser", "passwordCredentials" : { "username" : "qauser", "password" : "r1q150Sqa" } } } jul 19, 2017 10:22:39 PM org.glassfish.jersey.filter.LoggingFilter log INFORMAÇÕES: 1 Client response received on thread main 1 < 200 1 < Connection: Keep-Alive 1 < Content-Length: 3589 1 < Content-Type: application/json 1 < Date: Wed, 19 Jul 2017 22:22:50 GMT 1 < Keep-Alive: timeout=5, max=100 1 < Server: Apache/2.4.6 (CentOS) mod_wsgi/3.4 Python/2.7.5 1 < Vary: X-Auth-Token 1 < x-openstack-request-id: req-750bb4e6-1e4f-4c0c-993a-2be18b048a02 22:22:40.047 [main] DEBUG org.openstack4j.core.transport.internal.HttpExecutor - Executing Request: http://cinderdev-ny.qa.os.com:8776/v2/7685011104884466bd89286f411d518d -> /volumes/detail jul 19, 2017 10:22:40 PM org.glassfish.jersey.filter.LoggingFilter log INFORMAÇÕES: 1 Sending client request on thread main 1 > GET http://cinderdev-ny.qa.os.com:8776/v2/7685011104884466bd89286f411d518d/volumes/detail 1 > Accept: application/json 1 > User-Agent: OpenStack4j / OpenStack Client 1 > X-Auth-Token: gAAAAABZb9u6QyKkBpO-cGjqeU7bfdA0c7NI1pxsbdbu9x-YSANSycg336dYdugPWfpcmypDX0i_NeYqCXntWpr8aeLBjiX-1Ci9KuUxQqgZ6OeyCEA7xCdA-2Yph0ZmfizmY2gpv6Clu88fQRiUZ5-pwRL8wDx_cxbHXP31cpIslrCHXO9kjyc jul 19, 2017 10:22:40 PM org.glassfish.jersey.filter.LoggingFilter log INFORMAÇÕES: 1 Client response received on thread main 1 < 200 1 < Connection: keep-alive 1 < Content-Length: 7824 1 < Content-Type: application/json 1 < Date: Wed, 19 Jul 2017 22:22:51 GMT 1 < X-Compute-Request-Id: req-9682967d-a64c-4988-a956-1533109f169b 1 < X-Openstack-Request-Id: req-9682967d-a64c-4988-a956-1533109f169b {"volumes":[{"attachments":[],"links":[{"href":"http://cinderdev-ny.qa.os.com:8776/v2/7685011104884466bd89286f411d518d/volumes/66e7a82d-9333-4036-8377-e7d36c3bc4a8","rel":"self"},{"href":"http://cinderdev-ny.qa.os.com:8776/7685011104884466bd89286f411d518d/volumes/66e7a82d-9333-4036-8377-e7d36c3bc4a8","rel":"bookmark"}],"availability_zone":"TVTDEV","encrypted":false,"updated_at":"2017-07-06T14:18:14.000000","replication_status":null,"snapshot_id":null,"id":"66e7a82d-9333-4036-8377-e7d36c3bc4a8","size":40,"user_id":"059c54a608f946719ed63e2795a5f152","os-vol-tenant-attr:tenant_id":"7685011104884466bd89286f411d518d","metadata":{"readonly":"False"},"status":"available","volume_image_metadata":{},"description":"","multiattach":false,"source_volid":null,"consistencygroup_id":null,"name":"volume2","bootable":"false","created_at":"2017-07-06T14:14:07.000000","volume_type":null},{"attachments":[{"server_id":"42a02167-bf3c-4e1a-a7ef-240f0c03bb43","attachment_id":"01dd0352-045e-489f-9fa1-25a8dd4e95d5","attached_at":"2017-07-06T13:45:43.000000","host_name":null,"volume_id":"1fd4026b-7df8-4214-bf11-e066d4c288c8","device":"/dev/vda","id":"1fd4026b-7df8-4214-bf11-e066d4c288c8"}],"links":[{"href":"http://cinderdev-ny.qa.os.com:8776/v2/7685011104884466bd89286f411d518d/volumes/1fd4026b-7df8-4214-bf11-e066d4c288c8","rel":"self"},{"href":"http://cinderdev-ny.qa.os.com:8776/7685011104884466bd89286f411d518d/volumes/1fd4026b-7df8-4214-bf11-e066d4c288c8","rel":"bookmark"}],"availability_zone":"TVTDEV","encrypted":false,"updated_at":"2017-07-06T13:45:43.000000","replication_status":null,"snapshot_id":null,"id":"1fd4026b-7df8-4214-bf11-e066d4c288c8","size":20,"user_id":"059c54a608f946719ed63e2795a5f152","os-vol-tenant-attr:tenant_id":"7685011104884466bd89286f411d518d","metadata":{"readonly":"False","attached_mode":"rw"},"status":"in-use","volume_image_metadata":{"checksum":"9a8351980a29d7865e24db96a9ea610c","min_ram":"1","disk_format":"raw","image_name":"Ubuntu-16.04","image_id":"fdb80d73-8f34-46e7-8ce8-1c2c3f2fbaa4","container_format":"bare","min_disk":"10","size":"2361393152"},"description":"","multiattach":false,"source_volid":null,"consistencygroup_id":null,"name":"","bootable":"true","created_at":"2017-07-06T13:45:35.000000","volume_type":null}]}

maven depency:

<dependency>
   <groupId>org.pacesys</groupId>
    <artifactId>openstack4j-core</artifactId>
    <version>3.0.4</version>
</dependency>

<dependency>
    <groupId>org.pacesys.openstack4j.connectors</groupId>
    <artifactId>openstack4j-jersey2</artifactId>
    <version>3.0.4</version>
</dependency>
auhlig commented 7 years ago

The cinder v3 call is going to http://192.168.55.34:5000/v3/volumes/detail, which is the keystone port. It's not getting the correct endpoint from the service catalog, which should be 

{
    "endpoints": [{
            "region_id": "NYQA",
            "url": "http://cinderdev-ny.qa.os.com.br:8776/v3/7685011104884466bd89286f411d518d",
            "region": "NYQA",
            "interface": "admin",
            "id": "a550810eba0d43b280ec1872265e7188"
        },
        ...
    ]
}
rafael-bertoli commented 7 years ago

This is a bug or I need to change the code? The code used to obtain volumes is:

OSClientV3 os = OSFactory.builderV3()
                    .endpoint("http://192.168.55.34:5000/v3")
                    .credentials("qauser", "r1q150Sqa", Identifier.byName("default"))
                    .scopeToProject(Identifier.byName("qauser"), Identifier.byName("default"))
                    .authenticate();
// I've tried various combinations of credentials and scope

List<? extends Volume> volumes = os.blockStorage().volumes().list();

I expected sdk to transparently resolve the use of endpoints...

rafael-bertoli commented 7 years ago

I solved the problem by changing the "forName" method of the ServiceType enum to the following code:

public static ServiceType forName(String name) {
    if (name == null || name.isEmpty()) return ServiceType.UNKNOWN;

    for (ServiceType s : ServiceType.values()) {
        if (name.toLowerCase().startsWith(s.getServiceName().toLowerCase())) {
            return s;
        }
        if (name.toLowerCase().startsWith(s.name().toLowerCase())) {
            return s;
        }
        if (name.toLowerCase().startsWith(s.type.toLowerCase())) {
            return s;
        }
    }
    return ServiceType.UNKNOWN;
}

The attribute "name" it was "volumev3" and this enum don't has this value. This way you can correct future issues with version number.

Edit: The release of the openstack I am using is the ocata.