githubixx / ansible-role-longhorn-kubernetes

Ansible role to install Longhorn cloud native distributed block storage for Kubernetes
https://www.tauceti.blog/posts/kubernetes-the-not-so-hard-way-with-ansible-persistent-storage-part-2/
6 stars 1 forks source link

Need to add become: root to ubuntu.yml? #8

Open toastedcrumpets opened 5 months ago

toastedcrumpets commented 5 months ago

First, thank you for a wonderful ansible role, just trying out deploying it now. Installing longhorn to a single-node remote Ubuntu 22.04 server with microk8s I had some issues. In particular I needed to modify tasks/ubuntu.yml to add become: yes to the multipathd tasks, otherwise I have permission errors.

- name: Ensure multipathd conf.d directory
  become: yes
  ansible.builtin.file:
    path: "{{ longhorn_multipathd_blacklist_directory }}"
    state: directory
    mode: "{{ longhorn_multipathd_blacklist_directory_perm }}"
    owner: "root"
    group: "root"

- name: Ensure multipathd blacklist for Longhorn
  become: yes
  when:
    - longhorn_multipathd_blacklist_directory
    - longhorn_multipathd_blacklist_file
  ansible.builtin.copy:
    src: "files/{{ longhorn_multipathd_blacklist_directory }}/{{ longhorn_multipathd_blacklist_file }}"
    dest: "/{{ longhorn_multipathd_blacklist_directory }}/{{ longhorn_multipathd_blacklist_file }}"
    mode: "{{ longhorn_multipathd_blacklist_file_perm }}"
    owner: "root"
    group: "root"
  notify:
    - restart multipathd

If I try adding become: yes to when the role is invoked, then I get other issues as my root user doesn't have a kubeconf setup.

Can you suggest the best way to handle permissions for the multipath? Is this something needed more generally? Thank you again for all your hard work.

githubixx commented 5 months ago

Hi there! Thanks! :smiley: Regarding "become": What I normally do is setting

ansible_user: "ansible"
ansible_become: true
ansible_become_method: "sudo"

in ether host_vars/ for the host where I need it or in group_vars/all.yml when it should basically be true for all hosts (can be overridden again in host_vars/ for every host or in Ansible's hosts file. Here is an example: https://www.tauceti.blog/posts/virtualization-with-archlinux-qemu-kvm-part-2/#setup-ansibles-host-file In that case the become: yes in your example shouldn't be needed anymore. The user ansible above has sudo permissions. So Ansible is logging in as this user when connecting to the remote host.

The Ansible kubernetes.* modules are also respect K8S_AUTH_KUBECONFIG environment variable. E.g. https://github.com/githubixx/ansible-role-kubernetes-worker/blob/master/molecule/default/converge.yml#L36-L37 There you can specify where your "kubeconfig" file is located.

Hope this helps a bit. BTW: I'm currently updating the role to Longhorn 1.5.3. Should be ready the next days. I'm currently testing the changes.

toastedcrumpets commented 5 months ago

Hi, Thanks for the fast reply! Yes your approach works, I just thought it was "best practice" to to avoid escalation into root wherever possible, which is why I wanted to put become only on the tasks that need it. I've managed to follow what you've done and its all working now. So perhaps this is a request to add "become: yes" on those two tasks, but I'm more than happy to work around this, the role is so extremely helpful!

I hit two other issues that I'd thought I'd mention:

  1. When running in template mode, line 78 of defaults/main.yml, i.e. longhorn_template_output_directory: "{{ '~/longhorn/template' | expanduser }}" seems to evalulate on the ansible control node; however, I'm using longhorn_delegate_to a remote node which has helm etc installed, but doesn't have the same username, leading to a failure at that stage. Do you expect this to work? Or is it undefined behaviour to run with longhorn_delegate_to != 127.0.0.1?
  2. I'm using microk8s on Ubuntu server, and this needs "csi.kubeletRootDir: "/var/snap/microk8s/common/var/lib/kubelet". I currently add this in longhorn_values_user.yml.j2, but would be happy to try to write a pull request to add this as an auto-detected configuration change. Is this something you would want?

Thank you again, I realise I'm just dumping in this issue, but if you want me to work on something I will make a separate issue/pull request. I've now got everything working on my cluster and you've made it so much easier and declarative so thank you!

githubixx commented 5 months ago

So perhaps this is a request to add "become: yes" on those two tasks,

I was already thinking about re-factoring the role as you suggested so that become: yes will be just added where it's needed. I just haven't had time yet. And it's not just the two tasks because the is also "update" and "delete" tasks besides "install" that also needs to be adjusted.

When running in template mode, line 78 of defaults/main.yml, i.e. longhorn_template_output_directory: "{{ '~/longhorn/template' | expanduser }}" seems to evalulate on the ansible control node;

You're right. delegate_to and the default of longhorn_template_output_directory doesn't really match as it uses the current user on the Ansible Control node (which is normally the one that runs ansible-playbook) and puts the file into the directory of that user on the remote host. This is of course not desired. I guess I need to change the default value to something else. The current default it not a very good one.

I'm using microk8s on Ubuntu server, and this needs "csi.kubeletRootDir: "/var/snap/microk8s/common/var/lib/kubelet". I currently add this in longhorn_values_user.yml.j2, but would be happy to try to write a pull request to add this as an auto-detected configuration change. Is this something you would want?

Of course. As long as it works reliable I'm fine with it. I also want to implement a change where one can specify the location of longhorn_values_user.yml.j2 file. That it has to be place in the template/ directory of the role is not really great.