Metal Instances - Githubissues

JacquesMrz commented 3 years ago

As a user, I want to start a Baremetal instance (no virtualisation) in my Public Cloud tenant, I want to have the choice between monthly billing and pay as you go model (hourly billing).

cambierr commented 3 years ago

Do you have any more details on the available instances type ? as well as pricing model compared to the same server taken as monthly bare metal ? Also, would it benefit from network isolation like VM does, which other bare metal server doesn't ?

mhurtrel commented 3 years ago

@cambierr Nothing precise yet but my colleague @JacquesMrz may be able to share more later. I can say that most of our current customer demand is for medium size worker nodes, corresponding to the equivalent to the 64GB // 1Gbps private connectivity // 2x NVme variant of the Advance 2 : https://www.ovhcloud.com/en/bare-metal/advance/adv-2/

We also plan to offer the very large baremetal machines that we use to offer the different VM models (during the year they will be updated, but you can get a rough idea of the specs by looking at the largest VMs models of each line, that are "one VM per host" currently). There will be private network compatibility, but I am not sure of the details

I you have a wishlist, here is a great place to do it :) . If you plan to use some of those baremetal as managed worker nodes in K8S please fill the survey here : https://labs.ovh.com/gpu-baremetal-kubernetes-nodes

cambierr commented 3 years ago

We indeed use those servers in a K8S deployment but a self-managed one (due to OVH's managed one not yet able to do multi region, vrack networking, etc)

Right now we have to use a mix of bare metal (for databases due to the public cloud poor storage performances) and public cloud (for scaled applicative workloads).

Based on this, I'd love to see INFRA-2 servers usable in the public cloud since they deliver the most value-for-the-buck for database workloads: they major issue we have with those right now being that 1) they are not integrated in neutron's networking, leading to a server-side VLAN choice, potentially allowing intrusion in other networks and 2) they are not nicely integrated in OpenStack provisioning that we use a lot (through Rancher & RKE): would this solution based on OpenStack Ironic ?

mhurtrel commented 3 years ago

Perfect, Infra-2 corresponds typically to other requests we have and target to answer first ! Yes that will be based on Ironic :)

hbrombeer commented 3 years ago

Is it still possible to join Alpha or Beta? Requested it a month ago and didnt got an answer

cambierr commented 3 years ago

Indeed; I'd too be interested in spending some time testing this solution but not especially for GPU nodes; more for infra-like nodes :)

desaintmartin commented 3 years ago

One use case that would be interesting is having Infra3 or 4 maxed-out (huge CPU, 256/512GB RAM, either fast and big NVMe storage or Cinder network storage for better Kube integration) for big PostgreSQL master-slaves clusters connected to a managed Kubernetes cluster.

mhurtrel commented 3 years ago

Those machines would be a bit more expensive in public cloud pricing, and I anticipate that those type of use case (Postgresql cluster) is intended for a long term use. Did you know you can already buy those personnalised machines in the baremetal (aka dedicated servers) universe and connect them to a Kubernetes cluster through OVHcloud multiregion private network "vRack" ? https://github.com/ovh/public-cloud-roadmap/issues/15

hbrombeer commented 3 years ago

Do you have an ETA for those Baremetal machines? We applied a few months ago and didnt got into the Testing Period. @mhurtrel

mhurtrel commented 3 years ago

@hbrombeer unfortunately, we do not have a precise date for this, as we decided to offer this baremetal machine on a totally new distributed network mesh, to offer extremely competitive performance/price. But this came with a significant delay. We think that we will be able to open back the baremetal beta with a small number of machines on the current infrastructure in Spring or summer in a specific openstack region, and offer this commercially late 2021.

JacquesMrz commented 3 years ago

English below.

Bonjour à tous,

Vous rêvez de pouvoir démarrer du Baremetal en une poignée de secondes via l’API Openstack et de pouvoir le consommer à l’heure ? Bonne nouvelle, Baremetal as a Service est disponible en version BETA dans la région Public Cloud GRA9 !

Une seule flavor sera disponible en quantité limitée, et le service sera gratuit le temps de la BETA. A l’issu de la beta tous les services seront coupés et les données supprimées.

Pour accéder au Service : Remplir ce formulaire : https://survey.ovh.com/index.php/443698?lang=en Vous recevrez un mail vous informant que votre projet Public Cloud peut accéder au service Activer la région GRA9 pour votre projet Public Cloud si ce n’est pas déjà fait

Tout ce qu’il y a à savoir au sujet de cette BETA se trouver ici : https://labs.ovh.com/bare-metal-on-public-cloud

On compte sur vous pour nous partager vos feedback, via : Cloud ML : cloud@ml.ovh.net Gitter : ovh/BareMetal-aaS Github : https://github.com/ovh/public-cloud-roadmap/issues/42

La team Public Cloud.

Coucou all,

Your dream is to be able to start Baremetal in a few seconds via the Openstack API, having it hourly billed ? Good news, Baremetal as a Service is available in BETA version in GRA9 Public Cloud region!

A unique flavor will be available in limited quantities, and the service will be free for the duration of the BETA. At the end of the beta all services will be terminated and the data will be deleted.

To access the service: Fill in this form: https://survey.ovh.com/index.php/443698?lang=en You will receive an email informing you that your Public Cloud project can access the service Activate the GRA9 region for your Public Cloud project if it is not already done

Everything you need to know about this BETA can be found here: https://labs.ovh.com/bare-metal-on-public-cloud

We count on you to share your feedback, via : Cloud ML : cloud@ml.ovh.net Gitter : ovh/BareMetal-aaS Github : https://github.com/ovh/public-cloud-roadmap/issues/42

The Public Cloud team.

cambierr commented 3 years ago

Hi @JacquesMrz ! I'm sorry to ask but ... how different is it from the IRONIC lab that was open more than one year ago ?

hugoatease commented 3 years ago

Hi @JacquesMrz,

In an attempt to deploy a Red Hat Satellite test instance on the BETA Bare Metal as a Service, I tried to build a custom image of RHEL 7 for use with the metal.eg-32 instances.

I’ve successfully been able to import and boot a custom image of RHEL 8 with a KVM Guest Image and the following commands :

os image create --disk-format=qcow2 --private --file Downloads/rhel-8.3-update-2-x86_64-kvm.qcow2 rhel-8.3 os image set --property 'boot_info={"kernel_location":{"kernel_path":"/boot/vmlinuz-4.18.0-240.15.1.el8_3.x86_64","initrd_path":"/boot/initramfs-4.18.0-240.15.1.el8_3.x86_64.img","fsuuid":"fcd1c8db-7a38-49db-a7e6-ab45e6c819d6"},"cmdline":"BOOT_IMAGE=(hd0,gpt3)/boot/vmlinuz-4.18.0-240.15.1.el8_3.x86_64 root=UUID=fcd1c8db-7a38-49db-a7e6-ab45e6c819d6 ro console=tty0 console=ttyS0,115200n8 no_timer_check net.ifnames=0 crashkernel=auto","src_kernels":["4.19.0-5-amd64","4.19.0-9-amd64","4.19.0-10-amd64"]}' rhel-8.3

In order to find what to use in the boot_info property, I first booted a production instance (with type s1-4), and constructed the boot_info with the following params :

• kernel_path: path of the vmlinuz image on /boot • initrd_path : path of the initramfs on /boot • fsuuid : blkid, then get the UUID of the /boot partition • cmdline : copy paste of the /proc/cmdline contents.

Pretty-printed boot_info is :

{
    "kernel_location": {
        "kernel_path": "/boot/vmlinuz-3.10.0-1160.el7.x86_64",
        "initrd_path": "/boot/initramfs-3.10.0-1160.el7.x86_64.img",
        "fsuuid": "367cfd1d-b581-48b3-9e5e-9ee13c5a670a"
    },
    "cmdline": "BOOT_IMAGE=/boot/vmlinuz-3.10.0-1160.el7.x86_64 root=UUID=367cfd1d-b581-48b3-9e5e-9ee13c5a670a ro crashkernel=auto rhgb quiet LANG=en_US.UTF-8",
    "src_kernels": [
        "4.19.0-5-amd64",
        "4.19.0-9-amd64",
        "4.19.0-10-amd64"
    ]
}

It’s working well for RHEL 8 KVM images, which includes an EFI partition. RHEL 7 images doesn’t include this partition, so I imported an ISO image of the installer in Glance, setted the hw_firmware_type=uefi to the resulting image, did an EFI install of RHEL 7 on an attached volume to the installer (s1-4) instance, installed cloud-init, created an image of the resulting volume, setted the boot_info parameters and tried to boot a metal.eg-32 instance on it.

Resulting bare metal instance seems unreachable. A classic instance launched with the hw_firmware_type=uefi property works fine.

When I did the first process of importing a pre-built image of RHEL 7 for KVM with boot_info populated, the first boot worked well, but the instance was lost after the first reboot. I understand that the metal hypervisor reads the boot_info for the provisioning step and first boot, then needs an EFI configured system in order to be able to handle reboots. Am I missing something ?

Custom images are marked as supported on https://labs.ovh.com/bare-metal-on-public-cloud. Do you have a shareable process for image builds on the metal hypervisor ?

Regards, and thanks for the great work bringing metal instances to life !

hugoatease commented 3 years ago

Following https://github.com/ovh/public-cloud-roadmap/issues/42#issuecomment-870324121, I've been able to create a CentOS 7 ISO suitable for use with metal.eg-32 instances, by cloning the OVH provided Centos 7 - UEFI image and adding this boot_info property to it :

{
    "kernel_location": {
        "kernel_path": "/boot/vmlinuz-3.10.0-1160.25.1.el7.x86_64",
        "initrd_path": "/boot/initramfs-3.10.0-1160.25.1.el7.x86_64.img",
        "fsuuid": "37fc6117-f9a9-4838-8122-d7d44845100f"
    },
    "cmdline": "BOOT_IMAGE=/vmlinuz-3.10.0-1160.25.1.el7.x86_64 root=UUID=546e8e23-1f3f-4792-829a-387ba6652cf0 ro console=ttyS0,115200 crashkernel=auto LANG=en_US.UTF-8",
    "src_kernels": [
        "4.19.0-5-amd64",
        "4.19.0-9-amd64",
        "4.19.0-10-amd64"
    ]
}

I've used the following commands :

openstack image save --file centos-uefi.img 'Centos 7 - UEFI'
openstack image create --container-format=bare --disk-format=raw --private --file=centos-uefi.img --progress centos-7-baremetal
openstack image set --property 'boot_info={"kernel_location":{"kernel_path":"/boot/vmlinuz-3.10.0-1160.25.1.el7.x86_64","initrd_path":"/boot/initramfs-3.10.0-1160.25.1.el7.x86_64.img","fsuuid":"37fc6117-f9a9-4838-8122-d7d44845100f"},"cmdline":"BOOT_IMAGE=/vmlinuz-3.10.0-1160.25.1.el7.x86_64 root=UUID=546e8e23-1f3f-4792-829a-387ba6652cf0 ro console=ttyS0,115200 crashkernel=auto LANG=en_US.UTF-8","src_kernels":["4.19.0-5-amd64","4.19.0-9-amd64","4.19.0-10-amd64"]}' centos-7-baremetal

jcosmao commented 3 years ago

Hi @hugoatease

Seems you succeed to bring up your custom image on CentOS7.

I understand that the metal hypervisor reads the boot_info for the provisioning step and first boot, then needs an EFI configured system in order to be able to handle reboots. Am I missing something ?

Exactly, boot_info is only used at first boot after deploy to speed up deployment using kexec. But it's optional, it's just used to spare a hard reboot. So image should work without this property. And yes, baremetal flavor need an EFI partition.

When I did the first process of importing a pre-built image of RHEL 7 for KVM with boot_info populated, the first boot worked well, but the instance was lost after the first reboot.

So my guess is you succeed to build a functional image (kexec OK after deploy) but there were probably a misconfiguration in boot chain (maybe wrong block device naming in fstab, grub... as image has been built from qemu instance). It's not simple to troubleshoot without a console access.

Custom images are marked as supported on https://labs.ovh.com/bare-metal-on-public-cloud. Do you have a shareable process for image builds on the metal hypervisor ?

We don't have a documentation on how to build a baremetal image yet, but we'll try to provide it.

Thx for your feedback

matmicro commented 2 years ago

What is the ETA targeted for GA ?

Any idea when the Baremetal flavor will be available into MKS for beta testing ?

matmicro commented 1 year ago

Any idea when the Baremetal flavor will be available into MKS for beta testing ?

JacquesMrz commented 1 year ago

Hello all,

The first range of Metal Instances will be available in the coming two weeks. This range will propose flavors with a Balanced ratio CPU/RAM.

The flavors will be available day one in the following locations: BHS / SBG / GRA / DE / UK / WAW

Available features at launch: Software RAID Cloud-init Cold Snapshot Private network (only default vlan = vlan0) Console Access

Regards !

LoonyRules commented 1 year ago

That pricing will be reasons why it's not widely used.

Edit: I pressed create comment too quickly, here's some extra stuff I wanted to add.

How is that hourly and monthly pricing going to work? At 0.5€/hr, that's 340 hours (14 days) before reaching the 170€/mo price. Is the hour billing going to be capped at 340 hours per month so it's always 170€/mo? Or will it actually cost 340€/mo to have it for 28 days?

MrDienns commented 1 year ago

That pricing will be reasons why it's not widely used.

Edit: I pressed create comment too quickly, here's some extra stuff I wanted to add.

How is that hourly and monthly pricing going to work? At 0.5€/hr, that's 340 hours (14 days) before reaching the 170€/mo price. Is the hour billing going to be capped at 340 hours per month so it's always 170€/mo? Or will it actually cost 340€/mo to have it for 28 days?

The latter is the case for regular public cloud already. Safe to assume you can either pay monthly and pay €170, or pay hourly, which results in €340 for 28 days. So paying on a monthly term makes absolutely zero sense since the exact same specs bare metal cost like half as much. This bare metal cloud thing is only profitable if you need dedicated server power like less than 7 days a month.

I don't see what the benefit of this is besides faster delivery times and no installation cost. And faster delivery times / no installation costs aren't worth double the price.

wbulot commented 1 year ago

Hello, According to this page https://labs.ovh.com/bare-metal-on-public-cloud the bare metal instances are supposed to be equipped with 10gb nic. However, they are currently in 1gb, and the official release seems to remain on 1gb. Is it planned to release 10gb instances? We are interested in bm-l1 but without the 10gb network it becomes completely useless for us. Thanks

JacquesMrz commented 1 year ago

Hi all,

Metal instances are live in the following Public Cloud regions: GRA11 / SBG7 / UK1 / DE1 / WAW1 / BHS5.

More features to come, more flavors to come, all will be described in dedicated issues.

Feedbacks are welcome !

MaxHayman commented 1 year ago

@JacquesMrz Where do you want feedback as the ticket is closed?

I was suprised that I couldn't add a Pool of these to my Managed Kubernetes like we can with all other of the instance types.

Will this be a feature and when?

mhurtrel commented 1 year ago

@MaxHayman concering Managed Kubernetes Service integration, you can follow this : https://github.com/ovh/public-cloud-roadmap/issues/18 Note that we are actively exploring options, though the current limitations of baremetal nodes are inviting us to contact soon the people that follow this issue to decide which integration options would make the most sense.

gilou commented 1 year ago

Hi, I missed that issue completely, I have been a very happy beta tester of Ironic… Feedback is said to be welcome, so I'll go for it (and I'm happy I found the price range today as well). What came back from my tests:

console access, at least for image baking: it's quite messy not to be able to have serial / KVM access… at least to bake images, or to install custom OS. One of our usage is for tryouts of baremetal solutions, and those are not easy to set up on a cold image…
10 Gbps network (at least private) option : kind of the same use case here, sometimes we want to set up network-heavy systems to support beta test or training material, it'd be good to have 10G connectivity between servers (not necessarily guaranteed, and certainly not over DCs / Internet…)

Also, it's great that it can be used almost transparently over the OpenStack API, really. It enables nice scenarios for hybrid setups, and guarantees actual reversability, and that is a good thing!

LisaPerrier commented 1 year ago

Hello everyone, As the new Product Owner for Managed Kubernetes Service and Managed Private Registry, I am glad to share with you this survey for Metal Instances on Managed Kubernetes. It will only take a few minutes to complete and your feedback is invaluable for us. We would appreciate it if you could take the time to fill it out.   The survey can be accessed through this link : https://survey.ovh.com/index.php/127286 . (French language is also available)    Thank you for your support and we look forward to hearing your thoughts. 😄

ovh / public-cloud-roadmap

Metal Instances #42