operator-framework / operator-sdk

SDK for building Kubernetes applications. Provides high level APIs, useful abstractions, and project scaffolding.
https://sdk.operatorframework.io
Apache License 2.0
7.23k stars 1.75k forks source link

Definition of Hosts #5827

Closed caatclaudia closed 2 years ago

caatclaudia commented 2 years ago

Context

I'm creating a Kubernetes operator, through Ansible, that allows me to monitor the node's state from a Kubernetes Cluster, namely that allows me to copy a directory to a remote host. For this, my main.yml file executes the following commands:

- name: Copy files to Remote host
  hosts. <IP>
  tasks:
    - name: Copy
      become: true
      copy:
        src: ~/Downloads/files/
        dest: ~/Downloads/files/
        owner: <user>
        group: <group>        
        mode: 0777        

When I use these commands, the pod cannot execute the commands correctly, as it does not show output for these commands, since I didn't define any host and I can't use the ip directly in the file, am I right?

make docker-build docker-push IMG=<DOCKER USER>/<REPOSITORY>:latest
make deploy IMG=<DOCKER USER>/<REPOSITORY>:latest
kubectl get deployment -n <NAMESPACE>
kubectl create -f config/samples/<FILE NAME>.yaml -n <NAMESPACE>
kubectl get pods -n <NAMESPACE>
kubectl logs --follow <POD ID> -n <NAMESPACE>

Question

How can I resolve this issue? Where can I set this host? I can't find any operator directories that allow me to do this. Can I do it directly in this task file?

caatclaudia commented 2 years ago

@everettraven Sorry, do you think you could help me here? I think that is a dumb question, but I really don't know, sorry

everettraven commented 2 years ago

Sorry, do you think you could help me here? I think that is a dumb question, but I really don't know, sorry

@caatclaudia No dumb questions here, and I am always happy to help however I can.

One thing I noticed is that your hosts definition in the playbook is using a . after it instead of a : (at least in the snippet you included). I'm not sure if this would cause the issue you are experiencing, but as far as I understand it you should be able to use normal playbook definitions to perform operations in your operator.

@asmacdo or @fabianvf is it possible to set the hosts definition within the playbook with the Ansible operators?

caatclaudia commented 2 years ago

Sorry, the . was just a copy mistake, in my tasks.yaml I have it right. The error that I see is the following: image

My entire file is the following:

---

- name: Creating a file with content
  copy:
    dest: "./data.sh"
    content: |
      #!/bin/sh
      awk '$3=="kB"{if ($2>1024^2){$2=$2/1024^2;$3="GB";} else if ($2>1024){$2=$2/1024;$3="                                                                                                  MB";}} 1' /proc/meminfo | column -t | grep MemAvailable | awk '{printf "%.2f\n", $2}'
      df -h | awk '$NF=="/"{printf "%d\n", $5}'
      cat /proc/stat |grep cpu |tail -1|awk '{print ($5*100)/($2+$3+$4+$5+$6+$7+$8+$9+$10)}                                                                                                  '|awk '{print " " 100-$1}'

- name: Output
  shell: "sh ./data.sh"
  register: output

- name: Verify Memory Usage
  shell: "sh ./data.sh | head -n 1"
  register: memoryUsage_output

- name: Verify Disk Usage
  shell: "sh ./data.sh | sed -n '2p'"
  register: diskUsage_output

- name: Verify CPU Usage
  shell: "sh ./data.sh | sed -n '3p'"
  register: CPUload_output

- name: Information
  vars:
    msg: |
        Memory Available: {{ memoryUsage_output.stdout }} MB
        Disk Usage: {{ diskUsage_output.stdout }} %
        CPU Load: {{ CPUload_output.stdout }} %

  debug:
    msg: "{{ msg.split('\n') }}"

- debug: msg="Is not necessary to migrate services"
  when:
    - memoryUsage_output.stdout | int > 200
    - diskUsage_output.stdout | int < 75
    - CPUload_output.stdout | int < 75

#- meta: end_play
#  when:
#    - memoryUsage_output.stdout | int > 200
#    - diskUsage_output.stdout | int < 75
#    - CPUload_output.stdout | int < 75

- debug: msg="Is necessary to migrate services"

- name: Change Nodes Pods
  hosts: 192.168.1.125
  tasks:
    - name: Change
      lineinfile:
        path: /etc/Downloads/files/*
        line: "  nodeName: node01"

- name: Apply Files
  shell: "kubectl replace --force -f /etc/Downloads/files/*"

- debug: msg="Files have been copied and applied"
everettraven commented 2 years ago

@caatclaudia So it seems to me after going through some of the Ansible documentation that you will need to have an inventory that contains that ip address to use it as a host in the playbook file.

I think this means you will need to create an inventory/hosts file in the playbook/group_vars directory so that it is copied over with the playbook into the Docker image.

Documentation on building an inventory can be found here: https://docs.ansible.com/ansible/latest/user_guide/intro_inventory.html

This is all speculation as I haven't actually done this myself (when I have some extra time I will try and do something similar to see if I can get it to work), so let me know if this helps!

caatclaudia commented 2 years ago

Ok, I created a playbook and hosts files and my playbook is able to look to this file. But I am having this issue when I try to execute commands on my remote machine image

My Dockerfile as this commands, but the issue continues:

USER root

RUN dnf install -y sshpass

And I am able to get the version of sshpass image

Any suggestion?

everettraven commented 2 years ago

@caatclaudia I did a little digging and this seems to be related to an Ansible default of creating a temp directory in ~/.ansible/tmp. It seems like by default ansible tries to use the $HOME directory of the user on the remote machine where the user is the same user that ran the Ansible playbook. In your case I think it is likely that the user being used in your container (i believe the user is ansible) doesn't exist on the remote machine.

Here is a link to a Stack Overflow question that had a similar issue: https://stackoverflow.com/questions/35176548/authentication-or-permission-failure-did-not-have-permissions-on-the-remote-dir

I think you may also need to create an ansible.cfg file that gets copied over to the container. This config file should include setting the value remote_tmp to be something like /tmp or another temp directory that it would have permissions to write to.

Here is a link to the Ansible docs for configuration settings: https://docs.ansible.com/ansible/latest/reference_appendices/config.html

It also looks like the container for running ansible operators already has an ansible.cfg file in the /etc/ansible/ directory so you may also be able to make modifications to that file in the Dockerfile if that seems like it would be easier than creating another one to be used.

The /etc/ansible/ creation for the base image can be seen here: https://github.com/operator-framework/operator-sdk/blob/280335593c76c1fb14ce96100938f140ddb42659/images/ansible-operator/base.Dockerfile#L12-L16

fabianvf commented 2 years ago

@asmacdo or @fabianvf is it possible to set the hosts definition within the playbook with the Ansible operators?

Yeah it should totally be possible to target remote hosts within a playbook. So long as the host address or IP is resolvable + reachable from inside the pod running in kubernetes it should behave exactly as ansible running outside the cluster. If you want to target the host via SSH you'll need to include your ssh keys (or use sshpass and password authentication as it seems you are doing) etc in the container, and you'll have to make sure that you're configured to log in as the appropriate use on the remote host that you are targeting. Also just to cut out some complexity it's good to verify the playbook is working properly from outside of kubernetes.

@everettraven sounds like you're on the right path

varshaprasad96 commented 2 years ago

@caatclaudia Hope the comments help in solving the issue. Closing this for now, please feel free to reopen!