Ice doesn't support region-scoped EC2 reserved instances

[patrickvalenzuela]

This is a relatively new type of reserved instance (https://aws.amazon.com/blogs/aws/ec2-reserved-instance-update-convertible-ris-and-regional-benefit/), and it looks like Ice hasn't been updated since this addition.

Before this change, reserved instances always had an associated availability zone. Now, they have a scope which is either "Region" or "Availability Zone" (the zone is defined in this latter case).

FYI: I tried to hack together a fix (my company has this type of reserved instances), but didn't get very far. At minimum, I believe this requires updating aws-java-sdk (which in turn requires updates to httpcore, httpclient, and potentially other libraries). This also requires some code changes to java/com/netflix/ice/processor/ReservationCapacityPoller.java and java/com/netflix/ice/basic/BasicReservationService.java in order to process the new RIs.

After making some modifications, I still can't get these RIs and their associated costs reflected in the Ice reader. I believe there may be some additional logic needed here since Ice wasn't designed with these RIs in mind.

[nfonrose]

Based on your comment, I understand that you have managed to fix the Exceptions that Ice was running into. But then you don't see the correct prices reflected for your instances, is that right ?

If that's the case, you probably need to look inside the BasicLineItemProcessor (line 218), where it looks for the price associated with a particular RI. This is probably where the problem is lying (it can't find the price information in its mapping tables).

Also, in order to help us help you, can you post the original Stacktrace (maybe the Stacktraces) you run into when you initially tried to run Ice against the new RI types ?

[patrickvalenzuela]

@nfonrose you're correct with your assessment. I've fixed the Exceptions but now don't get the right prices.

After updating the aws-java-sdk/httpcore/httpclient lib versions and fixing missing JSON imports in src/java/com/netflix/ice/basic/EddaResourceService.java:, the first stack trace is below. This happens because getAvailabilityZone() is called on a ReservedInstances instance representing a region-scoped RI and it has no such value. I made some modifications, but wasn't able to get the pricing for these instances.

Thanks for the tip re: BasicLineItemProcessor; I'll check that next.

2016-12-22 21:40:33,425 [com.netflix.ice.processor.ReservationCapacityPoller] INFO  processor.ReservationCapacityPoller  - read <redacted> reservations.
| Error 2016-12-22 21:40:40,262 [com.netflix.ice.processor.ReservationCapacityPoller] ERROR processor.ReservationCapacityPoller  - Error polling
Message: null
   Line | Method
->> 333 | hash                  in java.util.concurrent.ConcurrentHashMap
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
|   988 | get                   in     ''
|   130 | getZone . . . . . . . in com.netflix.ice.tag.Zone
|   417 | updateEc2Reservations in com.netflix.ice.basic.BasicReservationService
|   170 | poll . . . . . . . .  in com.netflix.ice.processor.ReservationCapacityPoller
|    50 | doWork                in com.netflix.ice.common.Poller
|    28 | access$000 . . . . .  in     ''
|    88 | run                   in com.netflix.ice.common.Poller$1
^   745 | run . . . . . . . . . in java.lang.Thread```

[adamalex]

@patrickvalenzuela did you have any luck getting this to work? Are you willing to share your branch? I may be able to assist if there is some work left to do.