softlayer / softlayer-java

SoftLayer API Client for Java
https://softlayer.github.io/softlayer-java/
MIT License
24 stars 26 forks source link

SL-API issue pagination of virtual machines with masks #52

Open isrfig opened 6 years ago

isrfig commented 6 years ago

The issue we're facing is to retrieve the relational properties of a list of SoftLayer_Virtual_Guest using pagination. We use masks to get the relational properties like power state, os, datacenter data etc.... of a virtual machine as there are many vms in a particular account the api throws an out of memory error, the service.getObject method ignores pagination, so I used service.getVirtualGuests() which the pagination works but only brings the local properties of the vm, if I put a mask to get the relational properties in the service, service.getVirtualGuests() thows another error, one way I found to do this is use service.getVirtualGuests() to get the basic info and then use Guest.Service virtualGuestsService = Guest.service(client, guest.getId()); with a mask to get the relational properties for each vm in a loop, but this will generate thousands of api calls, so Im not sure if this is the best/correct approach, the best would be if I could use service.getVirtualGuests() with the mask to get the relational properties, attached is a sample code. vm-pagination.txt

acamacho82 commented 6 years ago

Yeah I got the same behavior, there is no way to set resultLimits and masks in order to be used in the same method.

Result limit is being ignored with the following code:

service.setResultLimit(new ResultLimit(offset, limit));
service.withMask().virtualGuests().powerState().name();
List<Guest> listGuests = service.getObject().getVirtualGuests();

And on this case, when removing getObject(), the object-mask doesn't work.

service.setResultLimit(new ResultLimit(offset, limit));
service.withMask().virtualGuests().powerState().name();
List<Guest> listGuests = service.getVirtualGuests();

It seems that it could be fixed here #19

@isrfig the workaround you are using isn't the best on this case but I think it is still correct. In other cases this approach is the best solution because some relational properties have a lot of data and/or the mask is to long in depth (for example, to retrieve detailed information of items of all invoices in the account)

allmightyspiff commented 6 years ago

This may just be a bug with the java-client, I think the SLAPI itself supports resultLimits and objectMasks.

I'll mark this as a bug for future development. I don't have an ETA, so your work around might be best.

Just keep in mind there is a rate limit imposed by the API, so you may have to watch out for a rate limit exception and be prepared to sleep for a second before continuing.

allmightyspiff commented 6 years ago

@isrfig instead of using withMask() you can set the object mask manually with setMask(string) which can be used to work around this bug until we figure out a more clever solution.

ApiClient client = new RestApiClient().withCredentials(user, apiKey);
((RestApiClient) client).setLoggingEnabled(true);
Account.Service accountService = Account.service(client);
accountService.setResultLimit(new ResultLimit(10));

accountService.setMask("mask[id, hostname, powerState[name]]");
    for (Guest bd : accountService.getVirtualGuests()) {
    System.out.println("BD: " + bd.getId() + " - " + bd.getHostname());
}
camporter commented 6 years ago

The result limit is not being ignored in the initial case. The result limit applies to the base object being queried. In the case of a getObject request, it won't do anything, as the virtual guests are masked off the account which is result limited, not the virtual guests.

The string-based mask is an available workaround for now.