search

firecracker-microvm / firecracker

Secure and fast microVMs for serverless computing.
http://firecracker-microvm.io
Apache License 2.0
26.36k stars 1.83k forks source link

vGPU? #2487

Closed S4nfs closed 3 years ago

S4nfs commented 3 years ago

Is this hypervisor support Gpu passthrough or Nvidia vgpus to VMs?

sandreim commented 3 years ago

Hi @S4nfs! Right now there is no support for passthrough in Firecracker and no explicit plans for adding it as a general purpose feature. We are however considering different options to enable hardware accelerated inference capabilities for serverless workloads, see https://github.com/firecracker-microvm/firecracker/issues/1179.

sandreim commented 3 years ago

We are eager to see some details about your specific usecase for GPUs.

S4nfs commented 3 years ago

I am planning on to build a cloud computing platform (like paperspace) with my colleague and for that its necessary to build it from scratch. We are using Citrix Hypervisor right now but they are providing vGPU feature only with their premium edition 100$/month, which is way too expensive for us.

That's why we are looking for an open source nested virtualization that support vGPU passthrough. At this time the only options left are KVM and RHV.

sandreim commented 3 years ago

I am planning on to build a cloud computing platform (like paperspace) with my colleague and for that its necessary to build it from scratch. We are using Citrix Hypervisor right now but they are providing vGPU feature only with their premium edition 100$/month, which is way too expensive for us.

Interesting project!

That's why we are looking for an open source nested virtualization that support vGPU passthrough. At this time the only options left are KVM and RHV.

Firecracker should work as a guest hypervisor, but our testing pipelines do not cover this usecase. Also, due to the fact that Firecracker emulates virtio devices, I/O will have a much higher penalty in a nested environment compared to running on bare metal. I would expect I/O overhead to go up by ~10x.

S4nfs commented 3 years ago

As for our project GPUs are very important but in AWS especially with bare metals there's no such information of how to deploy these with GPUs attached (such as Tesla T4 with i3 instance). If you know something about this in AWS could you please tell me if these instances even support GPUs.

sandreim commented 3 years ago

I understand the scenario you describe and I think this AWS page might help: https://aws.amazon.com/ec2/instance-types/ Please reopen if you have additional questions on #1179