search

FINRAOS / herd-mdl

Herd-MDL, a turnkey managed data lake in the cloud. See https://finraos.github.io/herd-mdl/ for more information.
Apache License 2.0
16 stars 14 forks source link

Launch MDL in any region #64

Closed nateiam closed 5 years ago

nateiam commented 6 years ago

Observed in MDL: Herd fails to launch EMR clusters in regions other than the default (us-east-1)

Steps to reproduce Install MDL in us-west-2 (or any other region)

Defective Behavior

Metastor (nested) stack fails to create because cluster creation fails. The underlying issue is that while trying to launch an EMR cluster to process Herd objects it looks for subnets in the us-east-1 region which are not available.

Desired Behavior

EMR cluster-create 'action' looks for subnets in the region where the stack is deployed.

kusid commented 6 years ago

Observed in MDL: Herd fails to launch EMR clusters in regions other than the default (us-east-1)

Steps to reproduce Install MDL in us-west-2 (or any other region)

Defective Behavior

Metastor (nested) stack fails to create because cluster creation fails. The underlying issue is that while trying to launch an EMR cluster to process Herd objects it looks for subnets in the us-east-1 region which are not available.

Desired Behavior

EMR cluster-create 'action' looks for subnets in the region where the stack is deployed.

kusid commented 6 years ago

Dev comments: With the proper fix available in Herd - we will upgrade the appropriate Herd version reference in the MDL CFT and attempt to install a new stack in us-west-2.