kubeconfig context being cached incorrectly

[leecalcote]

Description

Expected Behavior

Screenshots

Environment:

• OS: [e.g. Ubuntu]
• Browser: [e.g. Chrome, Safari]
• Version: [e.g. 22]
• Device: [e.g. laptop, iPhone 8]

---

[Optional] To Reproduce Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

[Optional] Additional Context

[tangledbytes]

@leecalcote @kumarabd I think this issue is because of meshery server's caching issue rather than meshery adapter.

How did I reproduce the issue?

1. Started meshery and its adapters using docker-compose (my compose file has network mode set to "host")
2. KinD cluster and Meshery Adapters were reachable
3. Add a new minikube k8s cluster (minkube start - docker as driver)
4. Delete the older config from the settings page (so that the new kube config is loaded on page reload)
5. Refresh the page:
   1. Adapters were no longer reachable¹
   2. Minikube k8s was not reachable²
6. Delete the auto loaded k8s config and upload the minified kubeconfig
7. Both adapters and k8s are reachable³
8. Switch Context back to KinD cluster
9. Remove the older config and refresh the page so that new config is automatically loaded
10. Now the K8s is reachable from meshery server but adapters are not⁴
11. Check logs of the adapters Error creating adapter instance.Error validating kubeconfig.open /home/utkarsh/.minikube/ca.crt: no such file or directory

My Conclusion It may seem that adapter has cached the kubeconfig but instead it is because the meshery server cached the kubeconfig at step 9. This kubeconfig is not minified and flattened but because the context is not minikube so meshery server couldn't care less about it. However meshery server sends this kubeconfig to the adapter where it tries to validate the config, there it complains that the certificate path is not reachable and hence throws an error.

How did the minikube config worked in step 7 then? It worked because the config was minified and flattened hence it had all of the certificate data in them.

How can we fix it? I am not sure if we should be fixing this because this seems like an appropriate behaviour. In order to avoid the error, we can definitely filter out the invalid certificate authority credentials (like we do for authentication info) but in that case we may end up with the issue that the current-context is referring to a cluster which no longer exists in the internal kubeconfig.

On the user's end, all they have to do is to upload a flattened kubeconfig and this issue should go away with that :sweat_smile:

---

1 - Adapter was not reachable because non minified and non flattened configs were being fed to the adapters 2 - K8s was not reachable because non minified and non flattened configs were being used while the context was still minikube 3 - Both were available because config is flattened 4 - Context has changed to KinD which has the valid creds hence k8s is reachable however kubeconfig validation fails on the adapter side because the config is not flattened.

[leecalcote]

High quality write-up. It takes this level of thought to make great software.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue is being automatically closed due to inactivity. However, you may choose to reopen this issue.

[leecalcote]

@kumarabd considerations to account for in Meshery Server, too.

@utkarsh-pro, Do you consider this a completed issue from the perspective of the adapters?

[kumarabd]

Un-assigning myself as this is covered in meshery server.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue is being automatically closed due to inactivity. However, you may choose to reopen this issue.