search

ii / infra

2 stars 4 forks source link

Multi-node cloudnative.coop cluster on Equinix DALLAS #4

Open hh opened 2 months ago

hh commented 2 months ago

Same as #1 sharing.io plus ability to fully template to new repo (likely new fork as cloudnative-coop/infra). The only changes necessary to deploy to Paris should be using a new DNS zone.

In addition, as time permits:

vielmetti commented 2 months ago

Good to see a setup plan and a teardown plan for this - certainly you will learn a lot from the exercise in #1 .

I will check latency from Paris to Amsterdam, we have a lot more gear / a lot more variety of gear in AM.

hh commented 2 months ago

@vielmetti do you want to recommend some gear / plans for us to target for Amsterdam / Paris?

hh commented 2 months ago

Here are the likely decent metros:

    metal facilities get \
       | grep Paris\\\|Amsterdam\\\|---\\\|METRO

    +----------------+------+-------+----------------------------------------------------+
    |      NAME      | CODE | METRO |                      FEATURES                      |
    +----------------+------+-------+----------------------------------------------------+
    | Paris          | pa4  | pa    | baremetal,backend_transfer,layer_2,global_ipv4,ibx |
    | Amsterdam      | am2  | am    | baremetal,backend_transfer,layer_2,global_ipv4,ibx |
    | Amsterdam      | am6  | am    | baremetal,layer_2,backend_transfer,global_ipv4,ibx |
    +----------------+------+-------+----------------------------------------------------+

Here are the available plan capacities that are not limited at the moment.

    metal capacity get --facilities pa4,am2,am6 | grep normal | sort --key=3 --field-separator="|"

    | am6      | a3.large.x86  | normal      |
    | pa4      | c2.medium.x86 | normal      |
    | am2      | c3.medium.x86 | normal      |
    | am6      | c3.medium.x86 | normal      |
    | am6      | m3.large.x86  | normal      |
    | pa4      | m3.large.x86  | normal      |
    | am6      | m3.small.x86  | normal      |
    | am6      | n2.xlarge.x86 | normal      |
    | am6      | n3.xlarge.x86 | normal      |
    | pa4      | n3.xlarge.x86 | normal      |
    | am6      | s3.xlarge.x86 | normal      |

Looking at these three larger ones, I'm leaning toward n3.xlarge.x86 if that works for you @vielmetti

n2.xlarge.x86 (384GB / 56 threads / 2x3.8TB NVME)

metal plan get --filter slug=n2.xlarge.x86 -o yaml | yq '.[].specs'
cpus:
  - arch: x86
    cores: "28"
    count: 2
    manufacturer: Intel
    model: "5120"
    name: Intel Xeon Gold 5120 Processor
    speed: 2.20GHz
    threads: "56"
    type: Intel Xeon Gold 5120 14-Core Processor @ 2.20GHz
drives:
  - category: boot
    count: 2
    size: 120GB
    type: SSD
  - category: storage
    count: 1
    size: 3.8TB
    type: NVME
features: {}
memory:
  total: 384GB
nics:
  - count: 4
    type: 10Gbps

n3.xlarge.x86 (512GB / 64 threads / 2x3.8TB NVME)

metal plan get --filter slug=n3.xlarge.x86 -o yaml | yq '.[].specs'
cpus:
  - arch: x86
    cores: "32"
    count: 1
    manufacturer: Intel
    model: 6314U
    name: Intel Xeon Gold 6314U Processor
    speed: 2.30GHz
    threads: "64"
    type: Intel Xeon Gold 6314U 32-Core Processor @ 2.30GHz
drives:
  - category: boot
    count: 2
    size: 256GB
    type: NVME
  - category: storage
    count: 2
    size: 3.84TB
    type: NVME
features:
  raid: true
  txt: true
memory:
  total: 512GB
nics:
  - count: 4
    type: 25Gbps

s3.xlarge.x86 (192GB / 48 threads / 2x240GB NVME + 12x8TB HDD)

metal plan get --filter slug=s3.xlarge.x86 -o yaml | yq '.[].specs'
cpus:
  - arch: x86
    cores: "24"
    count: 2
    manufacturer: Intel
    model: "4214"
    name: Intel Xeon Silver 4214 Processor
    speed: 2.20GHz
    threads: "48"
    type: Intel Xeon Silver 4214 24-Core Processor @ 2.20GHz
drives:
  - category: boot
    count: 2
    size: 960GB
    type: SSD
  - category: cache
    count: 2
    size: 240GB
    type: NVME
  - category: storage
    count: 12
    size: 8TB
    type: HDD
features:
  raid: true
  txt: true
memory:
  total: 192GB
nics:
  - count: 2
    type: 10Gbps
hh commented 4 weeks ago

This cluster needs to migrate away from Paris, now that Hong Kong is up, we'll likely migrate this one in Dallas or Silicon Valley.