Reprovision store nodes to address operational inconsistancies - Githubissues

CDLUC3 / mrt-doc

Documentation and Information regarding the Merritt repository

8 stars 4 forks source link

Reprovision store nodes to address operational inconsistancies #519

Open ashleygould opened 3 years ago

ashleygould commented 3 years ago

There are 4 hosts with fully qualified service name 'uc3-mrt-store-prd':

uc3-mrtstore01x2-prd
uc3-mrtstore02x2-prd
uc3-mrtstore03x2-prd
uc3-mrtstore04x2-prd

There are two operational issues involving these hosts:

issue: all four are in the same AWS AvailabiliyZone us-west-2c consequence:
- jeopardizes high-availability
- breaks automation staggered reboots during OS patching
issue: these four hosts actually provide 2 separate subservices: store-ingest, store-access consequence:
- two separate load balancer configurations
- operational confusion as to which store host does what

See full problem description and analysis in Pivotal Tracker: https://www.pivotaltracker.com/story/show/174605565

terrywbrady commented 3 years ago

I read your pivotal ticket, and I like the idea of differentiating the service names. I wonder if it would make sense to rename all 4 servers in the migration. What about something like this:

uc3-mrtstore-acc01x2-prd, mrtstore-acc02x2-prd
uc3-mrtstore-ing01x2-prd, mrtstore-ing02x2-prd