Awkward "permission denied" when trying to open symblinked Kube Config

[theAkito]

https://github.com/helm/helm/issues/11592

error: kubeconfig error: failed to load: failed to open file: open /home/user/.kube/config: permission denied

[ahmetb]

Is it only symlinked or is the symlink broken/not readable?

[theAkito]

@ahmetb

I use the Kube Config, the symlink points to, all the time. Works even with Helm, even though Helm shows an ugly warning.

Still, despite that fact, I have checked the link & functionality after running this plugin and it worked, as expected.

However, the plugin seems to check the permissions of the symlink, instead of the ones from the actual file behind it.

[ahmetb]

I don't think any of these tools check the permissions of the symlink. But they need to be able to open+read the symlink to find the actual path. Permissions of the symlink matter. That's just how symlinks work.

I am going to need a more clear explanation from you about what's not working and under what circumstances. You said the plugin has worked so I am confused now.

[theAkito]

Permissions of the symlink matter. That's just how symlinks work.

No, that's not how they work, though. The symlink's permissions does not matter at all, which is why it shows up as 777, when you look up its permissions.

The only way the permissions of the symlink itself matter, is if you are running macOS or some BSD.

I am going to need a more clear explanation from you about what's not working and under what circumstances. You said the plugin has worked so I am confused now.

I tried the plugin. Did not work.

Then, for this issue, I ran kubectl to check, whether the Kube Config permissions are fine. It worked without the plugin.

Again tried with plugin. Did not work, due to permission error.

Nothing works with the plugin. It won't run, because of permissions, which is objectively just wrong. It's a bug in the permission check's implementation.

[theAkito]

@ahmetb

I found the culprit.

https://github.com/ahmetb/kubectx/blob/3504e66edbe51e1bcbb57cd02b2cf6972e9d501d/internal/kubeconfig/kubeconfigloader.go#L39

This asks for too many permissions.

...
...
O_RDONLY [int](https://pkg.go.dev/builtin#int) = [syscall](https://pkg.go.dev/syscall).[O_RDONLY](https://pkg.go.dev/syscall#O_RDONLY) // open the file read-only.
....
O_RDWR   [int](https://pkg.go.dev/builtin#int) = [syscall](https://pkg.go.dev/syscall).[O_RDWR](https://pkg.go.dev/syscall#O_RDWR)   // open the file read-write.
...
...

Source

The file has to be opened with read permissions, only. No application like Helm, kubectl, or whatever is popular, needs write permissions.

Therefore, the permission requested has to be reduced to O_RDONLY.

However, when trying out the patch, which fixes this issue, I found out, that this plugin rewrites the Kube Config file on each change.

Which means, if you cannot fix the implementation due to Kubernetes, you have to update the documentation for this plugin, since it needs to state, that the Kube Config needs to be writable by the user accessing the Kube Config file.

By default, the Kube Config file does not need to be writable and therefore should not be writable. No other popular app I have tried, like Helm, needs a writable Kube Config file, which is why all other apps worked fine.

[ahmetb]

Understood. Kubectl actually updates the kubeconfig files if you run various "kubectl config" commands as well. Do they work fine when the file is a symlink? If so, we can also fix it here.

[theAkito]

Kubectl actually updates the kubeconfig files if you run various "kubectl config" commands as well. Do they work fine when the file is a symlink?

Well, as explained above, it's about the permission of the actual file behind the symbolic link(s).

The actual permission is read-only by default, so the file may not be modified.

For this purpose, I have tested renaming a context via kube config rename-context and it did not work. Though, this command does not seem to be common and I never needed it.

Anyway, the default permissions are something like 400, which would be read-only. Which in turn means, this isn't anything uncommon.

If so, we can also fix it here.

This issue can only be truly fixed, if there is a way for changing the context/namespace without actually modifying the Kube Config file on the filesystem.

If that is impossible, this issue can only be worked around by allowing more permissions to the Kube Config file, like adding write access.

Therefore, if it cannot be truly fixed, the documentation needs to be updated, so it is clear, that this plugin needs a writable Kube Config.

[ahmetb]

I don't think that can be fixed since the main purpose for this tool is to modify kubeconfig files.

And I am not inclined to document it either since there are not a lot of people doing this (this=symlinking kubeconfig path to a readonly file). It came up for the first time after all these years.

A lot of users set KUBECONFIG env var or still use writable files even if they use symlinks, so I think this is a corner case.

Also it's not very hard to debug as "ls" shows to any user it's a symlink, and they can see it's readonly.

[theAkito]

And I am not inclined to document it either since there are not a lot of people doing this (this=symlinking kubeconfig path to a readonly file).

By default, the Kube Config file does not need to be writable and therefore should not be writable. No other popular app I have tried, like Helm, needs a writable Kube Config file, which is why all other apps worked fine.

The actual permission is read-only by default, so the file may not be modified.

Anyway, the default permissions are something like 400, which would be read-only. Which in turn means, this isn't anything uncommon.

If that is impossible, this issue can only be worked around by allowing more permissions to the Kube Config file, like adding write access.

that this plugin needs a writable Kube Config.

Not sure how I can make it even more clear, that this is not related to symbolic links, at all. It's about the Kube Config file not having write permissions, which is the default.

It came up for the first time after all these years.

If people use the root user, then of course nobody will notice.

A lot of users set KUBECONFIG env var or still use writable files even if they use symlinks, so I think this is a corner case.

Again, not a symbolic link issue. It's a permission issue related to the Kube Config file, itself.

Also it's not very hard to debug as "ls" shows to any user it's a symlink, and they can see it's readonly.

Not sure how to make it more clear that symlinks don't have "permissions" the way you think. They are permissionless & show up as 777 when you ls -l it. The permissions are entirely based on what the actual file behind the symbolic links has. Symbolic links have no permissions on Linux.

they can see it's readonly.

Okay. Let me repeat for the 5th time.

Read-only is the default.

Read-write is not default.

Therefore, your plugin requires more access, than is usually needed.

Again, therefore, you need to document this, because it is important, that your plugin needs additional permissions, which are usually not needed. Kubernetes, Helm & all other popular tools run with a read-only Kube Config file.

One last time about the symbolic links, so we are on the same page, finally:

Symbolic links have no permissions on Linux. This is a Kube Config file permission issue. This is not an issue about symbolic linking.

[ahmetb]

By default, the Kube Config file does not need to be writable and therefore should not be writable. No other popular app I have tried, like Helm, needs a writable Kube Config file, which is why all other apps worked fine.

this is incorrect. Kubectl itself (when used with gcp, azure or oidc) store access tokens/refresh tokens in the kubeconfig file by updating the file.

I am not claiming the issue is with symlink permissions. But if you are a kubectl user and you get a permission error, it’s not too cryptic to see that the path referenced via the symlink is a readonly file (if that’s the case).

[ahmetb]

also just so you know

Symbolic links have no permissions on Linux.

This is not too relevant because symlinks do have permissions that matter on macOS. The semantics aren’t identical to Linux.

[ahmetb]

Again, therefore, you need to document this, because it is important

that’s what “you” think. As I said, this tool has a pretty significant user base and this came up for the first time over the past 5 years. So I am not inclined in documenting every single corner case caused by an odd set up —especially when running “ls” makes the situation clear— as something that everyone needs to read.

[theAkito]

this is incorrect. Kubectl itself (when used with gcp, azure or oidc) store access tokens/refresh tokens in the kubeconfig file by updating the file.

Yes. So, how does that matter, when you point out non-default setups? Irrelevant.

I am not claiming the issue is with symlink permissions. But if you are a kubectl user and you get a permission error, it’s not too cryptic to see that the path referenced via the symlink is a readonly file (if that’s the case).

You were talking the entire time about symlinks, when it was clear pretty early, that this is about permission issues on the Kube Config file, itself.

This is not too relevant because symlinks do have permissions that matter on macOS. The semantics aren’t identical to Linux.

This proves, you have neither read nor understood anything I had said.

I literally repeated this at least 5 times (including the one in the Helm Github Issue) and you still were talking about symlink permissions on Linux. 😄

You literally just copied what I said several times earlier. Like what? Why?

that’s what “you” think.

No. That's what reality "thinks", if reality had a mind.

I just gave you like 20 objective reasons, why your tool is out of the ordinary and this needs to be pointed out, as let others avoid a waste of time I have experienced. Hence, I created this issue, so this can be solved, even if it cannot be truly solved, because then we can warn people about this issue.

As I said, this tool has a pretty significant user base and this came up for the first time over the past 5 years.

Absolutely irrelevant. Does matter about 0%. There are tons of entire toolsets, which become obsolete, because something changed in upstream or a new feature has been included in upstream. One of the most famous examples is docker-compose. It was a separate project back then. So, just because "nothing came up" in all these years, it literally does not matter at all. Because, it came up just now!

I am not inclined in documenting every single corner case caused by an odd set up —especially when running “ls” makes the situation clear— as something that everyone needs to read.

Again, you have neither read nor understood a single thing I had said.

The case I was elaborating on is the DEFAULT case.

It's the DEFAULT.

The case you are covering is the actual edge case. You think, that the Kube Config file is normally readable. This is an edge case.

Whatever, as was evident along the entire conversation, you are neither willing nor able to accept actual facts representing the reality of how the Kube Config works by default. Not GCP or some weird edge case you are talking about. I am talking about the default setup. The default Kubernetes setup. Not some edge case cloud platform, making their own stuff.

P.S.:

ls does not show any permissions, except you run ls -l. Plus, it's never needed to check the permissions with any tool, except yours. So, your tool makes edge case assumptions.

You finally realised, that symlink permissions only matter on macOS and then go on about how it does not matter then. Actually, it does matter, because the largest user base is made out of Linux users. Nobody is running a Kubernetes server on macOS & only a minority manages Kubernetes from macOS. These brain acrobatics are astonishing.