search

2015-Middleware-Keynote / demo-ansible

Apache License 2.0
18 stars 24 forks source link

Provisioning to AWS us-west-1 doesn't work #144

Closed alberttwong closed 8 years ago

alberttwong commented 8 years ago

So I'm trying to provision the environment to us-west-1. Put in parameters.

alberts-MacBook-Pro:demo-ansible alwong$ ./run.py --no-confirm --cluster-id openshift --num-nodes 1 --keypair default --r53-zone alberttwong.com --rhsm-user alwong@redhat.com --rhsm-pass XXXX --region us-west-1 --ami ami-d1315fb1 --run-smoke-tests --num-smoke-test-users 5
Configured values:
    cluster_id: openshift
    ami: ami-d1315fb1
    region: us-west-1
    master instance_type: m4.large
    node_instance_type: m4.large
    infra_instance_type: m4.large
    keypair: default
    nodes: 1
    infra nodes: 1
    masters: 3
    console port: 8443
    api port: 8443
    deployment_type: openshift-enterprise
    hexboard_size: tiny
    r53_zone: alberttwong.com
    app_dns_prefix: apps
    host dns: openshift.alberttwong.com
    apps dns: apps.openshift.alberttwong.com
    rhsm_user: alwong@redhat.com
    rhsm_pass: *******
    num smoke users: 5
    default password: openshift3

 [WARNING]: provided hosts list is empty, only localhost is available

PLAY [Bootstrapping or Refreshing Environment] ********************************

TASK: [Validating options] ****************************************************
skipping: [localhost]

TASK: [Launch the CloudFormation Template] ************************************
failed: [localhost] => {"changed": true, "events": ["StackEvent AWS::CloudFormation::Stack openshift-demo-openshift ROLLBACK_COMPLETE", "StackEvent AWS::EC2::InternetGateway VpcInternetGateway DELETE_COMPLETE", "StackEvent AWS::EC2::VPC Vpc DELETE_COMPLETE", "StackEvent AWS::EC2::InternetGateway VpcInternetGateway DELETE_IN_PROGRESS", "StackEvent AWS::EC2::VPC Vpc DELETE_IN_PROGRESS", "StackEvent AWS::EC2::Volume LoggingVolume DELETE_COMPLETE", "StackEvent AWS::EC2::Volume MetricsVolume DELETE_COMPLETE", "StackEvent AWS::EC2::VPCGatewayAttachment VpcGA DELETE_COMPLETE", "StackEvent AWS::EC2::Volume RegistryVolume DELETE_COMPLETE", "StackEvent AWS::IAM::Role NodePolicy DELETE_COMPLETE", "StackEvent AWS::IAM::Role MasterPolicy DELETE_COMPLETE", "StackEvent AWS::EC2::SecurityGroup MasterExtElbSG DELETE_COMPLETE", "StackEvent AWS::EC2::SecurityGroup SshSG DELETE_COMPLETE", "StackEvent AWS::EC2::SecurityGroup NodeSG DELETE_COMPLETE", "StackEvent AWS::EC2::SecurityGroup InfraElbSG DELETE_COMPLETE", "StackEvent AWS::EC2::SecurityGroup InfraSG DELETE_COMPLETE", "StackEvent AWS::EC2::RouteTable VpcRouteTable DELETE_COMPLETE", "StackEvent AWS::EC2::SecurityGroup MasterIntElbSG DELETE_COMPLETE", "StackEvent AWS::EC2::SecurityGroup MasterSG DELETE_COMPLETE", "StackEvent AWS::EC2::SecurityGroup SshSG DELETE_IN_PROGRESS", "StackEvent AWS::EC2::SecurityGroup NodeSG DELETE_IN_PROGRESS", "StackEvent AWS::EC2::SecurityGroup MasterExtElbSG DELETE_IN_PROGRESS", "StackEvent AWS::EC2::SecurityGroup InfraSG DELETE_IN_PROGRESS", "StackEvent AWS::IAM::Role MasterPolicy DELETE_IN_PROGRESS", "StackEvent AWS::EC2::VPCGatewayAttachment VpcGA DELETE_IN_PROGRESS", "StackEvent AWS::EC2::Volume LoggingVolume DELETE_IN_PROGRESS", "StackEvent AWS::EC2::SecurityGroup MasterSG DELETE_IN_PROGRESS", "StackEvent AWS::EC2::Volume MetricsVolume DELETE_IN_PROGRESS", "StackEvent AWS::EC2::SecurityGroup MasterIntElbSG DELETE_IN_PROGRESS", "StackEvent AWS::EC2::SecurityGroup InfraElbSG DELETE_IN_PROGRESS", "StackEvent AWS::EC2::Volume RegistryVolume DELETE_IN_PROGRESS", "StackEvent AWS::EC2::RouteTable VpcRouteTable DELETE_IN_PROGRESS", "StackEvent AWS::EC2::Subnet Subnet1 DELETE_COMPLETE", "StackEvent AWS::IAM::Role NodePolicy DELETE_IN_PROGRESS", "StackEvent AWS::CloudFormation::Stack openshift-demo-openshift ROLLBACK_IN_PROGRESS", "StackEvent AWS::EC2::SecurityGroup InfraSG CREATE_FAILED", "StackEvent AWS::EC2::SecurityGroup SshSG CREATE_FAILED", "StackEvent AWS::EC2::SecurityGroup MasterSG CREATE_FAILED", "StackEvent AWS::EC2::SecurityGroup NodeSG CREATE_FAILED", "StackEvent AWS::EC2::SecurityGroup MasterExtElbSG CREATE_FAILED", "StackEvent AWS::EC2::SecurityGroup InfraElbSG CREATE_FAILED", "StackEvent AWS::EC2::SecurityGroup MasterIntElbSG CREATE_FAILED", "StackEvent AWS::IAM::Role MasterPolicy CREATE_FAILED", "StackEvent AWS::EC2::VPCGatewayAttachment VpcGA CREATE_FAILED", "StackEvent AWS::IAM::Role NodePolicy CREATE_FAILED", "StackEvent AWS::EC2::RouteTable VpcRouteTable CREATE_COMPLETE", "StackEvent AWS::EC2::VPCGatewayAttachment VpcGA CREATE_IN_PROGRESS", "StackEvent AWS::EC2::SecurityGroup SshSG CREATE_IN_PROGRESS", "StackEvent AWS::EC2::Subnet Subnet1 CREATE_FAILED", "StackEvent AWS::EC2::VPCGatewayAttachment VpcGA CREATE_IN_PROGRESS", "StackEvent AWS::EC2::SecurityGroup InfraSG CREATE_IN_PROGRESS", "StackEvent AWS::EC2::RouteTable VpcRouteTable CREATE_IN_PROGRESS", "StackEvent AWS::EC2::SecurityGroup MasterSG CREATE_IN_PROGRESS", "StackEvent AWS::EC2::SecurityGroup MasterExtElbSG CREATE_IN_PROGRESS", "StackEvent AWS::EC2::SecurityGroup InfraElbSG CREATE_IN_PROGRESS", "StackEvent AWS::EC2::SecurityGroup NodeSG CREATE_IN_PROGRESS", "StackEvent AWS::EC2::RouteTable VpcRouteTable CREATE_IN_PROGRESS", "StackEvent AWS::EC2::Subnet Subnet1 CREATE_IN_PROGRESS", "StackEvent AWS::EC2::SecurityGroup MasterIntElbSG CREATE_IN_PROGRESS", "StackEvent AWS::EC2::VPC Vpc CREATE_COMPLETE", "StackEvent AWS::EC2::Volume RegistryVolume CREATE_COMPLETE", "StackEvent AWS::EC2::Volume MetricsVolume CREATE_COMPLETE", "StackEvent AWS::EC2::Volume LoggingVolume CREATE_COMPLETE", "StackEvent AWS::EC2::InternetGateway VpcInternetGateway CREATE_COMPLETE", "StackEvent AWS::EC2::Volume MetricsVolume CREATE_IN_PROGRESS", "StackEvent AWS::EC2::VPC Vpc CREATE_IN_PROGRESS", "StackEvent AWS::EC2::Volume RegistryVolume CREATE_IN_PROGRESS", "StackEvent AWS::EC2::Volume LoggingVolume CREATE_IN_PROGRESS", "StackEvent AWS::IAM::Role MasterPolicy CREATE_IN_PROGRESS", "StackEvent AWS::IAM::Role NodePolicy CREATE_IN_PROGRESS", "StackEvent AWS::EC2::Volume MetricsVolume CREATE_IN_PROGRESS", "StackEvent AWS::EC2::Volume RegistryVolume CREATE_IN_PROGRESS", "StackEvent AWS::EC2::Volume LoggingVolume CREATE_IN_PROGRESS", "StackEvent AWS::EC2::VPC Vpc CREATE_IN_PROGRESS", "StackEvent AWS::EC2::InternetGateway VpcInternetGateway CREATE_IN_PROGRESS", "StackEvent AWS::EC2::InternetGateway VpcInternetGateway CREATE_IN_PROGRESS", "StackEvent AWS::CloudFormation::Stack openshift-demo-openshift CREATE_IN_PROGRESS"], "failed": true, "output": "Problem with CREATE. Rollback complete", "stack_outputs": {}}

FATAL: all hosts have already failed -- aborting

PLAY RECAP ********************************************************************
Launch the CloudFormation Template ------------------------------------- 98.05s
Validating options ------------------------------------------------------ 0.03s
           to retry, use: --limit @/Users/alwong/cloudformation_setup.retry

localhost                  : ok=0    changed=0    unreachable=0    failed=1

doesn't work... look in logs


2016-04-27  Status  Type    Logical ID  Status reason
17:36:59 UTC-0700   ROLLBACK_COMPLETE   AWS::CloudFormation::Stack  openshift-demo-openshift    
17:36:58 UTC-0700   DELETE_COMPLETE AWS::EC2::InternetGateway   VpcInternetGateway  
17:36:58 UTC-0700   DELETE_COMPLETE AWS::EC2::VPC   Vpc 
17:36:42 UTC-0700   DELETE_IN_PROGRESS  AWS::EC2::InternetGateway   VpcInternetGateway  
17:36:42 UTC-0700   DELETE_IN_PROGRESS  AWS::EC2::VPC   Vpc 
17:36:41 UTC-0700   DELETE_COMPLETE AWS::EC2::Volume    LoggingVolume   
17:36:40 UTC-0700   DELETE_COMPLETE AWS::EC2::Volume    MetricsVolume   
17:36:40 UTC-0700   DELETE_COMPLETE AWS::EC2::VPCGatewayAttachment  VpcGA   
17:36:40 UTC-0700   DELETE_COMPLETE AWS::EC2::Volume    RegistryVolume  
17:36:29 UTC-0700   DELETE_COMPLETE AWS::IAM::Role  NodePolicy  
17:36:28 UTC-0700   DELETE_COMPLETE AWS::IAM::Role  MasterPolicy    
17:36:26 UTC-0700   DELETE_COMPLETE AWS::EC2::SecurityGroup MasterExtElbSG  
17:36:26 UTC-0700   DELETE_COMPLETE AWS::EC2::SecurityGroup SshSG   
17:36:26 UTC-0700   DELETE_COMPLETE AWS::EC2::SecurityGroup NodeSG  
17:36:26 UTC-0700   DELETE_COMPLETE AWS::EC2::SecurityGroup InfraElbSG  
17:36:25 UTC-0700   DELETE_COMPLETE AWS::EC2::SecurityGroup InfraSG 
17:36:25 UTC-0700   DELETE_COMPLETE AWS::EC2::RouteTable    VpcRouteTable   
17:36:25 UTC-0700   DELETE_COMPLETE AWS::EC2::SecurityGroup MasterIntElbSG  
17:36:25 UTC-0700   DELETE_COMPLETE AWS::EC2::SecurityGroup MasterSG    
17:36:25 UTC-0700   DELETE_IN_PROGRESS  AWS::EC2::SecurityGroup SshSG   
17:36:25 UTC-0700   DELETE_IN_PROGRESS  AWS::EC2::SecurityGroup NodeSG  
17:36:25 UTC-0700   DELETE_IN_PROGRESS  AWS::EC2::SecurityGroup MasterExtElbSG  
17:36:25 UTC-0700   DELETE_IN_PROGRESS  AWS::EC2::SecurityGroup InfraSG 
17:36:25 UTC-0700   DELETE_IN_PROGRESS  AWS::IAM::Role  MasterPolicy    
17:36:24 UTC-0700   DELETE_IN_PROGRESS  AWS::EC2::VPCGatewayAttachment  VpcGA   
17:36:24 UTC-0700   DELETE_IN_PROGRESS  AWS::EC2::Volume    LoggingVolume   
17:36:24 UTC-0700   DELETE_IN_PROGRESS  AWS::EC2::SecurityGroup MasterSG    
17:36:24 UTC-0700   DELETE_IN_PROGRESS  AWS::EC2::Volume    MetricsVolume   
17:36:24 UTC-0700   DELETE_IN_PROGRESS  AWS::EC2::SecurityGroup MasterIntElbSG  
17:36:24 UTC-0700   DELETE_IN_PROGRESS  AWS::EC2::SecurityGroup InfraElbSG  
17:36:24 UTC-0700   DELETE_IN_PROGRESS  AWS::EC2::Volume    RegistryVolume  
17:36:24 UTC-0700   DELETE_IN_PROGRESS  AWS::EC2::RouteTable    VpcRouteTable   
17:36:24 UTC-0700   DELETE_COMPLETE AWS::EC2::Subnet    Subnet1 
17:36:24 UTC-0700   DELETE_IN_PROGRESS  AWS::IAM::Role  NodePolicy  
17:36:13 UTC-0700   ROLLBACK_IN_PROGRESS    AWS::CloudFormation::Stack  openshift-demo-openshift    The following resource(s) failed to create: [MasterPolicy, VpcGA, SshSG, Subnet1, InfraElbSG, NodeSG, MasterSG, MasterExtElbSG, MasterIntElbSG, InfraSG, NodePolicy]. . Rollback requested by user.
17:36:12 UTC-0700   CREATE_FAILED   AWS::EC2::SecurityGroup InfraSG Resource creation cancelled
17:36:12 UTC-0700   CREATE_FAILED   AWS::EC2::SecurityGroup SshSG   Resource creation cancelled
17:36:11 UTC-0700   CREATE_FAILED   AWS::EC2::SecurityGroup MasterSG    Resource creation cancelled
17:36:11 UTC-0700   CREATE_FAILED   AWS::EC2::SecurityGroup NodeSG  Resource creation cancelled
17:36:11 UTC-0700   CREATE_FAILED   AWS::EC2::SecurityGroup MasterExtElbSG  Resource creation cancelled
17:36:11 UTC-0700   CREATE_FAILED   AWS::EC2::SecurityGroup InfraElbSG  Resource creation cancelled
17:36:10 UTC-0700   CREATE_FAILED   AWS::EC2::SecurityGroup MasterIntElbSG  Resource creation cancelled
17:35:57 UTC-0700   CREATE_FAILED   AWS::IAM::Role  MasterPolicy    Resource creation cancelled
17:35:57 UTC-0700   CREATE_FAILED   AWS::EC2::VPCGatewayAttachment  VpcGA   Resource creation cancelled
17:35:57 UTC-0700   CREATE_FAILED   AWS::IAM::Role  NodePolicy  Resource creation cancelled
17:35:56 UTC-0700   CREATE_COMPLETE AWS::EC2::RouteTable    VpcRouteTable   
17:35:56 UTC-0700   CREATE_IN_PROGRESS  AWS::EC2::VPCGatewayAttachment  VpcGA   Resource creation Initiated
17:35:56 UTC-0700   CREATE_IN_PROGRESS  AWS::EC2::SecurityGroup SshSG   
17:35:56 UTC-0700   CREATE_FAILED   AWS::EC2::Subnet    Subnet1 Value (us-west-1a) for parameter availabilityZone is invalid. Subnets can currently only be created in the following availability zones: us-west-1b, us-west-1c.
17:35:55 UTC-0700   CREATE_IN_PROGRESS  AWS::EC2::VPCGatewayAttachment  VpcGA   
17:35:55 UTC-0700   CREATE_IN_PROGRESS  AWS::EC2::SecurityGroup InfraSG 
17:35:55 UTC-0700   CREATE_IN_PROGRESS  AWS::EC2::RouteTable    VpcRouteTable   Resource creation Initiated
17:35:55 UTC-0700   CREATE_IN_PROGRESS  AWS::EC2::SecurityGroup MasterSG    
17:35:55 UTC-0700   CREATE_IN_PROGRESS  AWS::EC2::SecurityGroup MasterExtElbSG  
17:35:55 UTC-0700   CREATE_IN_PROGRESS  AWS::EC2::SecurityGroup InfraElbSG  
17:35:55 UTC-0700   CREATE_IN_PROGRESS  AWS::EC2::SecurityGroup NodeSG  
17:35:55 UTC-0700   CREATE_IN_PROGRESS  AWS::EC2::RouteTable    VpcRouteTable   
17:35:55 UTC-0700   CREATE_IN_PROGRESS  AWS::EC2::Subnet    Subnet1 
17:35:54 UTC-0700   CREATE_IN_PROGRESS  AWS::EC2::SecurityGroup MasterIntElbSG  
17:35:50 UTC-0700   CREATE_COMPLETE AWS::EC2::VPC   Vpc 
17:35:50 UTC-0700   CREATE_COMPLETE AWS::EC2::Volume    RegistryVolume  
17:35:50 UTC-0700   CREATE_COMPLETE AWS::EC2::Volume    MetricsVolume   
17:35:50 UTC-0700   CREATE_COMPLETE AWS::EC2::Volume    LoggingVolume   
17:35:48 UTC-0700   CREATE_COMPLETE AWS::EC2::InternetGateway   VpcInternetGateway  
17:35:33 UTC-0700   CREATE_IN_PROGRESS  AWS::EC2::Volume    MetricsVolume   Resource creation Initiated
17:35:33 UTC-0700   CREATE_IN_PROGRESS  AWS::EC2::VPC   Vpc Resource creation Initiated
17:35:33 UTC-0700   CREATE_IN_PROGRESS  AWS::EC2::Volume    RegistryVolume  Resource creation Initiated
17:35:33 UTC-0700   CREATE_IN_PROGRESS  AWS::EC2::Volume    LoggingVolume   Resource creation Initiated
17:35:33 UTC-0700   CREATE_IN_PROGRESS  AWS::IAM::Role  MasterPolicy    
17:35:33 UTC-0700   CREATE_IN_PROGRESS  AWS::IAM::Role  NodePolicy  
17:35:33 UTC-0700   CREATE_IN_PROGRESS  AWS::EC2::Volume    MetricsVolume   
17:35:33 UTC-0700   CREATE_IN_PROGRESS  AWS::EC2::Volume    RegistryVolume  
17:35:33 UTC-0700   CREATE_IN_PROGRESS  AWS::EC2::Volume    LoggingVolume   
17:35:33 UTC-0700   CREATE_IN_PROGRESS  AWS::EC2::VPC   Vpc 
17:35:32 UTC-0700   CREATE_IN_PROGRESS  AWS::EC2::InternetGateway   VpcInternetGateway  Resource creation Initiated
17:35:31 UTC-0700   CREATE_IN_PROGRESS  AWS::EC2::InternetGateway   VpcInternetGateway  
17:35:25 UTC-0700   CREATE_IN_PROGRESS  AWS::CloudFormation::Stack  openshift-demo-openshift

seems like some regions don't like you creating subnets so I modify by run.py

Alberts-MacBook-Pro:demo-ansible alwong$ ./run.py --no-confirm --cluster-id openshift --num-nodes 1 --keypair default --r53-zone alberttwong.com --rhsm-user alwong@redhat.com --rhsm-pass XXXXXX --region us-west-1b --ami ami-d1315fb1 --run-smoke-tests --num-smoke-test-users 5
Configured values:
    cluster_id: openshift
    ami: ami-d1315fb1
    region: us-west-1b
    master instance_type: m4.large
    node_instance_type: m4.large
    infra_instance_type: m4.large
    keypair: default
    nodes: 1
    infra nodes: 1
    masters: 3
    console port: 8443
    api port: 8443
    deployment_type: openshift-enterprise
    hexboard_size: tiny
    r53_zone: alberttwong.com
    app_dns_prefix: apps
    host dns: openshift.alberttwong.com
    apps dns: apps.openshift.alberttwong.com
    rhsm_user: alwong@redhat.com
    rhsm_pass: *******
    num smoke users: 5
    default password: openshift3

 [WARNING]: provided hosts list is empty, only localhost is available

 [WARNING]: non fatal error while trying to template play variables: Failed to
template {{ lookup('ec2_zones_by_region', ec2_region) }}: Unexpected error in
during lookup: global name 'e' is not defined

PLAY [Bootstrapping or Refreshing Environment] ********************************

TASK: [Validating options] ****************************************************
skipping: [localhost]

TASK: [Launch the CloudFormation Template] ************************************
fatal: [localhost] => Failed to template {{ vpc_subnet_azs }}: Failed to template {{ lookup('ec2_zones_by_region', ec2_region) }}: Unexpected error in during lookup: global name 'e' is not defined

FATAL: all hosts have already failed -- aborting

PLAY RECAP ********************************************************************
Launch the CloudFormation Template -------------------------------------- 0.04s
Validating options ------------------------------------------------------ 0.01s
           to retry, use: --limit @/Users/alwong/cloudformation_setup.retry

localhost                  : ok=0    changed=0    unreachable=1    failed=0

it doesn't like it when I put in "us-west-1b" region.

kenthua commented 8 years ago

us-west-1b likely wouldn't work because it's a specific AZ in a region

Is checking for subnet support on an AZ even possible? us-west-1 has worked in the past, maybe something is happening on the AWS side. However if it rotates or is inconsistent that's not good. It can happen in other regions too.

detiber commented 8 years ago

@alberttwong as @kenthua mentioned it is because us-west-1b is an availability zone and not a region.

We don't currently expose a way to deploy to a specific AZ.

alberttwong commented 8 years ago

So the question is what is difference between my us-west-1 and us-west-2

thoraxe commented 8 years ago

You would need to explore you AWS settings to discover what differences, if any, exist.