search

hashicorp / nomad

Nomad is an easy-to-use, flexible, and performant workload orchestrator that can deploy a mix of microservice, batch, containerized, and non-containerized applications. Nomad is easy to operate and scale and has native Consul and Vault integrations.
https://www.nomadproject.io/
Other
14.76k stars 1.94k forks source link

[feature] add Amazon's Firecracker as a task driver #4923

Closed rushmorem closed 4 years ago

rushmorem commented 5 years ago

https://firecracker-microvm.github.io:

Firecracker is an open source virtualization technology that is purpose-built for creating and managing secure, multi-tenant containers and functions-based services

I've just discovered this gem from Amazon. I think having first class support for it in Nomad would be awesome!

yuriy-yarosh commented 5 years ago

I'm working on it.

dadgar commented 5 years ago

@yuriy-yarosh Just to be clear, Nomad 0.9.0 is bring drivers as plugins, so you should target the new style of drivers if you are working on this.

Unfortunately we don't have documentation yet on writing third party drivers but these links should be useful:

Essentially to build a custom driver, meet the driver interface and then create a main like the one I linked. You will then build that into a binary that you can place in a folder. Then run Nomad with the following flag: -plugin-dir=<path>. Nomad will then detect the plugin!

Hope this helps! Better docs will be coming as we get close to 0.9 releasing! Have fun!

yuriy-yarosh commented 5 years ago

Thanks, Alex, for being supportive.

I've dealt with hashicorp's stuff before, and as far as I know, I'll have to target stuff from the master branch right away. Surely, there are some undocumented things worth tinkering here and there, but overall, it's not that bad. I really doubt I'll caught myself in trouble with any existing hashicorp solutions, QA is simply good enough.

Hopefully, I'll be funded properly for this one... Wanna port some Amazon's solutions like API Gateway, Lambda, StepFunctions, SQS, SES, SNS, SWF so it would be possible to deploy a serverless stack locally, using HashiCorp solutions, for the development purposes, and then move to AWS when needed. Probably, it would be some rust, but I'll have to finish this one first.

Bringing AWS to the customer and the developers doesn't work that well right now, from the Amazon's perspective. I hope I'll be able to fix that.

Anyway, I've been struggling with k8s stuff lately and it's proven to be quite unreliable, so Nomad is the only option for me right now. I'm focusing some well HW-tuned deployments with decent HW utilization, not the best one, for sure - DPDK/SPDK and all that jazz.

yuriy-yarosh commented 5 years ago

Finished firecracker client for golang, it took more time to get a linux box with KVM :dango: than actually implementing it.

@dadgar what do you think about adding direct containerd support for nomad ? Docker driver by design became a bit obsolete...

c4milo commented 5 years ago

@yuriy-yarosh, that's a great idea. Many runtimes may be abstracted out by Containerd and Nomad's code and scope would simplify as a result. i.e: https://github.com/firecracker-microvm/firecracker-containerd

bithavoc commented 5 years ago

what's the official report for the driver @yuriy-yarosh, I found https://github.com/bitsnap/nomad-firecracker but no code.

yuriy-yarosh commented 5 years ago

@bithavoc I've finished it, but have been quite displeased with the firecracker design and some lack of features, so after a bit of tinkering I'm not considering firecracker a viable solution. I had started my own firecracker-ish KVM runner, but have no free time and resources to finish it right now - priorities had shifted a bit.

Surely, I could publish this driver, but it's not something I could recommend people to use.

Firecracker should and will be REFACTORED and partially REWRITTEN, and no one really knows when that day will come. So, there's just too much pointless hype going on around it.

pires commented 5 years ago

@yuri-yarosh can you please share what you came up with?

And maybe your thoughts on what parts of the design displease you, what features you're lacking, and so on. A blog post would be sweet 🙏

Last but not least, where did you find that Firecracker will be redesigned and rewritten?

yuriy-yarosh commented 5 years ago

@yuri-yarosh can you please share what you came up with?

Will do, but not not right now.

A blog post would be sweet 🙏

No one will pay me for that one, and I'm not a person who'll blindly contribute to a flawed product. So, why bother ?

My current engagements are much more meaningful and at least can shake up the market a bit.

maybe your thoughts on what parts of the design displease you

... worth hating just for the amount of hypocrisy and ignorance involved in that one

where did you find that Firecracker will be redesigned and rewritten?

Well, it's a no brainer if you'll search through and review the existing codebase.

I'd say that it's worth waiting few months 'till things settle down a bit, because right now Firecracker is too WIP.

yuriy-yarosh commented 5 years ago

Yup, it looks like Firecracker became something decent, at last. So, it's time to release the driver...

cneira commented 4 years ago

I just created a task driver for firecracker firecracker-task-driver, I'll submit it to the community drivers next week. Let me know if more features are needed, my use case is pretty simple at this point.

tgross commented 4 years ago

We've got that driver in our community driver docs. Going to close out this issue. Thanks!

github-actions[bot] commented 1 year ago

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.