rundeck-plugins / rundeck-ec2-nodes-plugin

Get resource node data from Amazon EC2
86 stars 45 forks source link

RUN-743: Integrate EC2 Node Source with multiple (or all) regions #117

Closed jsboak closed 2 years ago

jsboak commented 2 years ago

This PR enables a single node-source to integrate (pull in EC2's) from multiple regions by specifying multiple Endpoints using a comma-separated list of EC2 "endpoints" or by specifying ALL_REGIONS.

This is backwards compatible because the Endpoint field can still utilize a single entry.

To properly test, make sure the plugin retrieves nodes when the Endpoint field contains:

  1. A single endpoint (e.g. https://ec2.us-west-1.amazonaws.com)
  2. Multiple endpoints separated by a comma (with/without spaces after commas). Example: https://ec2.us-west-1.amazonaws.com, https://ec2.us-east-1.amazonaws.com,https://ec2.us-west-2.amazonaws.com
  3. ALL_REGIONS -> This will search for (and pull in) instances from all regions that the Credentials (or IAM Role) have access to.
  4. The Node Source fails and shows failure message when any of the entries in the Endpoint field are misspelled.

Multiple Endpoints: Screen Shot 2022-03-10 at 5 50 03 PM

Screen Shot 2022-03-10 at 5 49 16 PM

_ALLREGIONS

Screen Shot 2022-03-14 at 7 45 31 PM

Failure Message Screen Shot 2022-03-11 at 9 04 34 AM

fdevans commented 2 years ago

This may require #118 to complete successfully.

jsboak commented 2 years ago

@qualman want to take a look again and make sure all looks good?

qualman commented 2 years ago

Looking now!

qualman commented 2 years ago

Uh oh, another error :( Looks like it does not like ALL_REGIONS:

[2022-04-05T10:02:06,989] ERROR resources.ExceptionCatchingResourceModelSource - [ResourceModelSource: 3.source (aws-ec2), project: test]
java.lang.IllegalArgumentException: Endpoint does not contain a valid host name: https://ALL_REGIONS
        at com.amazonaws.AmazonWebServiceClient.computeSignerByURI(AmazonWebServiceClient.java:426) ~[aws-java-sdk-core-1.11.743.jar:?]
        at com.amazonaws.AmazonWebServiceClient.setEndpoint(AmazonWebServiceClient.java:318) ~[aws-java-sdk-core-1.11.743.jar:?]
        at com.dtolabs.rundeck.plugin.resources.ec2.InstanceToNodeMapper.performQuery(InstanceToNodeMapper.java:126) ~[?:?]
        at com.dtolabs.rundeck.plugin.resources.ec2.EC2ResourceModelSource.getNodes(EC2ResourceModelSource.java:258) ~[?:?]
        at com.dtolabs.rundeck.core.resources.ExceptionCatchingResourceModelSource.getNodes(ExceptionCatchingResourceModelSource.java:58) [rundeck-core-4.0.0-SNAPSHOT.jar:?]
qualman commented 2 years ago

Okay, mea culpa, I think I misread an error or something. I think it does in fact need to be getEndpoint().equals("ALL_REGIONS"), and that's the error here. I am going to test that real quick

qualman commented 2 years ago

Okay, @jsboak, I pushed a change that fixes that, and I tested locally and all is well. I am going to wait for tests and then we're ready to go