astro / microvm.nix

NixOS MicroVMs
https://astro.github.io/microvm.nix/
MIT License
1.38k stars 102 forks source link

how to start with my microvm.imperium #100

Closed luxus closed 1 year ago

luxus commented 1 year ago

hello, I'm still confused by how i should start. I saw skyflake and it sounds interesting too, but as a first step too complicated but i want to start with a host on a vmware esxi server and add some microvm's my idea is to have a micro VM for a reverse proxy and maybe later a WAF. the other microvm's should be small services that access shared storage for persistent files. in the best case, this storage can switch someday to a ceph storage.

so my first question how do I organize the files? from the talk, I saw I know that I have to start the vm's from the hostflake? because they have to be in the systemd to access virtfs?

i saw some clever stuff that works with git push in skyflake but I'm still confused how to start.. does anyone have a skeleton flake that does some of the stuff i want to do?

astro commented 1 year ago

Is the esx server running NixOS?

astro commented 1 year ago

If you mean VMware, that's supported by https://github.com/nix-community/nixos-generators

All of the five Hypervisors supported here require /dev/kvm which is provided by the Linux kernel.

luxus commented 1 year ago

from my understanding i can run a nixos virtual machine on vmware esxi. in this VM i have a normal linux kernel and use microvm's to deploy micro services like reverse proxy, git server, matrix server etc.

astro commented 1 year ago

Ok. Make sure to enable nested virtualization for your NixOS VM.

I think you can ignore Skyflake for now.

microvm.nix reuses the flake.nix way of defining nixosConfigurations. How you structure anything beyond that is entirely up to your liking!

The VMs don't need to be in the host flake.

What's a WAF? I am asking because it can get annoying having to redeploy the reverse-proxy whenever there's a new Vhost, or an additional backend instance.

luxus commented 1 year ago

waf its a web application firewall.. its a "better" reverseproxy yes u are right redeployment would be ugly.

I'm sure that I'm not the only person that is interested on a simple virtual machine flake. that is able to deploy micro vm's and access shared persistent space and the same nixstore

its really hard for me to come up with best practices right now

in that case with multiple flakes .. (host flakes and vms) a example would be such a help. i dunno why but i struggle with a lot of the very abstract readme's

astro commented 1 year ago

Did you already know the microvm.nix handbook besides the README? Perhaps we can turn this into an effort to improve it. Feel free to open a PR that litters the doc subdirectory with TODO notes for me to fill out.

Do you build your NixOS host from a flake?

luxus commented 1 year ago
Screenshot 2023-06-03 at 17 21 35

thats how i imagine my first step with microvm's

after reading the handbook and readme it's very focused on people that already fully understand a lot of nix and flakes.

I think microvm could be a good starting drug into the Nix world. but even after using nix on my Mac for years and on some machines for quite some time its tough to understand how to start with microvm.

maybe a cookbook section in the handbook would be good. a recipe for making dinner often includes washing the vegetables .. even every hobby chef know that.

sorry english is not my native language and its hard for me to explain my struggles :D if you want and have time we can hope on a quick call and i can explain it in german

astro commented 1 year ago

Ok, I'm thinking hard about how to make the handbook more beginner-friendly. Yet, it will stay specific to microvm.nix.

Fun fact: last year I considered writing a book about NixOS, containers, VMs and networking from my learnings with zentralwerk-network. I stopped because I find it hard to find a limit to what's in scope and what isn't. As an example, does it have to explain routing tables? Broader scope means more work but my time is limited.

luxus commented 1 year ago

like in cooking.. you maybe don't have to explain the fine details of cutting and washing vegetables. but maybe a direction where to find more info

i guess a well documented flake file would help a lot people to get into microvm details about how routing works is out of scope, my offer to hope on a quick call is still available. maybe we can work together on an example repo with documentation?

astro commented 1 year ago

Note that this issue is still open. I would love to improve the handbook to become more than just reference documentation.

I'm just not too fond of phone calls, especially if the result is going to be a document anyway. :-)

luxus commented 1 year ago

after no reaction about the phone thing i was guessing that this is not your cup of tea :D

I dunno what's the best way to approach this. maybe setting up a host with the microvms like in the drawing, and making documentation while creating it?