search

sergelogvinov / proxmox-csi-plugin

Proxmox CSI Plugin
Apache License 2.0
230 stars 19 forks source link

clarity of readme #192

Open breakingflower opened 2 months ago

breakingflower commented 2 months ago

Feature Request

improve clarity of readme for non-terraformed clusters

Description

hi

I am new to k8s / talos. I set up a talos cluster using the definitions on https://www.talos.dev/v1.7/talos-guides/install/virtualized-platforms/proxmox/.

I want to manage PV/PVCs on proxmox with this csi, but I'm having difficulties setting up the provider. I followed your instructions but I cant get it to work. I read on your other repo's that you terraform the cluster and generate some local snippets with zone/region/provider information. I did not terraform my setup, just following the guide. It is already configured with control plane and workers.

I specifically face troubles with the Overview part:

Can you please share insight into how to set this up manually, without using your terraform provider? I have tried to use the tf provider but I cant get it to work - many errors / issues that I don't have the knowledge to solve.

Thanks for understanding my limited knowledge.

sergelogvinov commented 1 month ago

Hi @mirceanton Can you help me with it.

Thanks.

vehagn commented 2 weeks ago

I wrote an article about how I got this working in my homelab here.

I had some difficulties to begin with as some of the terminology was new to me, but I tried explaining what I did in the article. Maybe I can try to have a go at updating the readme for this project as well.

sergelogvinov commented 2 weeks ago

@vehagn Your article is excellent; I truly enjoyed it. Thank you very much!

Probably we need to add ADOPTERS.md file to the repo, and share our experience/projects.

breakingflower commented 2 weeks ago

@vehagn Thank you so much. This article provided alot of clarity on the subject and actually it was very simple to do using your guide. There were just a few missing pieces with regards to the original readme which you complemented and improved very well.

One issue that I ran into was that you refer to the kubernetes.io/hostname: ctrl-00 which is the hostname of the talos node. If you use the deployment guide on the talos.dev page it will autogenerate names like talos-abc-cde as the hostname. I noticed that using this name, the pods were unable to create data because you get a 'matching vm not found' error. This hostname from my understanding needs to be the same as the name of the VM on your proxmox node. I set the hostname of the nodes in the proxmox gui to match the name of the VM (e.g. talos-ctrl-01). Now the provisioning works as intended.

A different way, although i did not test it yet, is to have a new kubernetes.io/proxmox-vm-name label that aligns with the proxmox vm name and use that to do the nodeselector: 

      nodeSelector:
        kubernetes.io/proxmox-vm-name: talos-work-02

I'm not sure what best practices are in k8s. It seems reasonable to have the hostname be generated so it's unique, and attach new labels to make sure you can do upgrades on your cluster. Can you share some insight?

sergelogvinov commented 2 weeks ago

Proxmox cloud is like manual cloud. Auto generated node-name very useful if you use node autoscaler. In terraform case better to set the node name in proxmox equal as in talos machine-config.

If you use Talos, try Talos CCM with Proxmox CCM. First one more powerful but you need to set properly metadata config. https://github.com/sergelogvinov/terraform-talos/blob/98916a2ae48a501c75b6d30cbdcddfd74a8da8e3/proxmox/instances-web.tf#L55-L72

sergelogvinov commented 2 weeks ago

@vehagn if you will have a time add you project to ADOPTERS.md (it already merged)

vehagn commented 2 weeks ago

@breakingflower I ran into the same issue the first time around, but found out I could template the machine config with 

machine:
  network:
    hostname: ${hostname}

as I'm doing here. I'm hardcoding the hostnames as I don't care much for auto-scaling. If I undestand @sergelogvinov's example correctly I think this is what he is doing as well. The more "correct" way would of course be to allow for automatic scaling of nodes.

I thought that I had made a point of making sure that the Proxmox VM name matches the actual hostname, but I forgot to actually make it!

@sergelogvinov Opened a PR adding myself to ADOPTERS.md. I hope it's OK that I linked the article. I'm in the process of cleaning up my Homelab GitHub repo where I'll start to use this awesome project.