Fresh K3s: Could not create API client: Invalid kube-config file

[adamhgriffith-uofu]

The default.config.yml file's download_kubeconfig:false causes the following error when installing on localhost for a single-node K3s cluster:

...
...
TASK [k8s_setup : Replace the kubeconfig cluster kube-api server IP with the public IP address, if the cluster is remote] *************
skipping: [ascender_host]

TASK [k8s_setup : Get a list of all nodes] ********************************************************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ansible_collections.kubernetes.core.plugins.module_utils.k8s.exceptions.CoreException: Could not create API client: Invalid kube-config file. No configuration found.
fatal: [ascender_host -> localhost]: FAILED! => {"changed": false, "msg": "Could not create API client: Invalid kube-config file. No configuration found."}

PLAY RECAP ****************************************************************************************************************************
ascender_host              : ok=12   changed=2    unreachable=0    failed=1    skipped=5    rescued=0    ignored=0   
localhost                  : ok=7    changed=0    unreachable=0    failed=0    skipped=4    rescued=0    ignored=0 

The related change from two months back is https://github.com/ctrliq/ascender-install/commit/e2bdd7e5eb24f24324270b3885e36343a4ee441e.

Changing to download_kubeconfig:true for the initial K3s install worked. Maybe the fix is to note this in ascender-install-instructions/k3s/README.md. To replicate, here is an example custom.config.yml:

---
# This variable specificies which Kubernetes platform Ascender and its components will be installed on.
k8s_platform: k3s
# Boolean indicating whether to set up a new k3s cluster (true) or use an existing k3s cluster (false)
kube_install: true
# Offline Install - Whether to use local assets to complete the install
k8s_offline: false
# Boolean indicating whether or not the kubeconfig file needs to be downloaded to the Ansible controller
download_kubeconfig: false
# Determines whether to use HTTP or HTTPS for Ascender and Ledger.
# If set to https, you MUST provide certificate/key options for the Installer to use.
k8s_lb_protocol: https
# Routable IP address for the K3s Master/Worker node
# required for DNS and k3s install
k3s_master_node_ip: "127.0.0.1"
# Boolean indicating whether to use the local /etc/hosts file for DNS resolution to access Ascender
use_etc_hosts: false
# TLS Certificate file location on the local installing machine
tls_crt_path: "/etc/letsencrypt/live/ascender/fullchain.pem"
# TLS Private Key file location on the local installing machine
tls_key_path: "/etc/letsencrypt/live/ascender/privkey.pem"
# A directory in which to place both temporary artifacts
# and timestamped Kubernetes Manifests to make Ascender/Ledger easy
# to uninstall
tmp_dir: "{{ playbook_dir}}/../ascender_install_artifacts"
# DNS resolvable hostname for Ascender service. This is required for install.
ASCENDER_HOSTNAME: ascender.our.domain
# Namespace for Ascender Kubernetes objects
ASCENDER_NAMESPACE: ascender
# Administrator username for Ascender
ASCENDER_ADMIN_USER: REDACTED
# Administrator password for Ascender
ASCENDER_ADMIN_PASSWORD: REDACTED
# The image tag indicating the version of Ascender you wish to install
ASCENDER_VERSION: 23.5.1
# The version of the AWX Operator used to install Ascender and its components
ANSIBLE_OPERATOR_VERSION: 2.9.0
# Determines whether to keep the secrets required to encrypt within Ascender (important when backing up)
ascender_garbage_collect_secrets: false
# External PostgreSQL database name used for Ascender (this DB must exist)
ascender_replicas: 2
# The Ascender web container image pull policy (If unsure, choose IfNotPresent)
image_pull_policy: IfNotPresent
# Determines whether or not Ledger will be installed
LEDGER_INSTALL: true
# DNS resolvable hostname for Ledger service. This is required for install
LEDGER_HOSTNAME: ledger.our.domain
# Number of replicas for the Ledger web container
ledger_web_replicas: 1
# Number of replicas for the Ledger Parser container
ledger_parser_replicas: 1
# The image tag indicating the version of Ledger you wish to install
LEDGER_VERSION: latest
# The Kubernetes namespace in which Ledger objects will live
LEDGER_NAMESPACE: ledger
# Admin password for Ledger (the username is admin by default)
LEDGER_ADMIN_PASSWORD: REDACTED
# Password for Ledger database
LEDGER_DB_PASSWORD: REDACTED

[kmf]

Related?: https://github.com/ctrliq/ascender-install/issues/63

[adamhgriffith-uofu]

Yes,

This looks to be the same issue. In the linked commit above, maybe the task when statement needs to read:

- name: Copy kubeconfig file from default location to the ~/.kube directory"
  ansible.builtin.fetch:
    src: /etc/rancher/k3s/k3s.yaml
    dest: ~/.kube/config
    flat: true
  become: true
  when:
  - ansible_host != "localhost"
  - download_kubeconfig

[michaelford85]

Happy to take a look at this.

[michaelford85]

So, after looking into this issue, we may need to be more clear about the function of the download_kubeconfig variable. when installing a fresh k3s cluster, you'll very likely want to download the kubeconfig file to its default location that kubectl expects (~/.kube/config). Whether your cluster is running on localhost, or a remote location, k3s will store it at /etc/rancher/k3s/k3s.yaml. The conditional of ansible_host != "localhost" is not necessary.

The reason that the download_kubeconfig variable exists is for situations where you already have the kubeconfig file in ~/.kube/config, and are running the installer against an existing k3s cluster (usually when upgrading ledger or ascender). I think a good option here may be to change the conditional to download_kubeconfig is true OR kube_install is true.

[kmf]

@michaelford85 can we please look at https://github.com/ctrliq/ascender-install/issues/65 tagged release might help

[michaelford85]

Acknowledging this question and will respond.