kubernetes-sigs / hierarchical-namespaces

Home of the Hierarchical Namespace Controller (HNC). Adds hierarchical policies and delegated creation to Kubernetes namespaces for improved in-cluster multitenancy.
Apache License 2.0
607 stars 105 forks source link

order in subnamespaceanchor deletion #344

Closed wibed closed 10 months ago

wibed commented 10 months ago

issue subnamespaces with a mount point to be deleted before others, to prevent blocking finalizers freezing the command

adrianludwin commented 10 months ago

What do you mean by "mount point"?

On Thu, Nov 2, 2023 at 3:33 AM wibed @.***> wrote:

issue subnamespaces with a mount point to be deleted before others, to prevent blocking finalizers freezing the command

— Reply to this email directly, view it on GitHub https://github.com/kubernetes-sigs/hierarchical-namespaces/issues/344, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE43PZH7LGM6BX5WU75F6C3YCNEDXAVCNFSM6AAAAAA62NV7JSVHI2DSMVQWIX3LMV43ASLTON2WKOZRHE3TGNRXGA4TMMQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

wibed commented 10 months ago

i meant for namespaces that contain resources such as pvc

adrianludwin commented 10 months ago

Can you describe a scenario where this happens? A tree of subnamespaces should be deleted from the root on down, and I thought PVCs could only be used from within their own namespace, so there shouldn't be any cross-namespace dependencies, or at least that's what I thought. If you could describe exactly what happens, and in what order, that would be helpful. Thanks!

On Thu, Nov 2, 2023 at 2:54 PM wibed @.***> wrote:

i meant for namespaces that contain resources such as pvc

— Reply to this email directly, view it on GitHub https://github.com/kubernetes-sigs/hierarchical-namespaces/issues/344#issuecomment-1791360399, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE43PZBNZ32YFYBJODSUW3LYCPT57AVCNFSM6AAAAAA62NV7JSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOJRGM3DAMZZHE . You are receiving this because you commented.Message ID: @.***>

wibed commented 10 months ago

of course heres is an example:

staging
├── [s] cert-manager
├── [s] csi-driver-nfs
├── [s] docker-registry
├── [s] gitsrv
├── [s] kubernetes-replicator
├── [s] metallb-system
└── [s] traefik

docker-registry and gitsrv do have a pvc mounted managed by csi-driver-nfs, i must have been overworked at the moment this presented itself to me as a problem, because i should use csi-driver-nfs outside of the staging namespace.

my bad! thank you for your time

adrianludwin commented 10 months ago

Oh, I see - it's not the PVC itself being used from the wrong namespace, it's the CSI driver. So you're concerned that if csi-driver-nfs is deleted before docker-registry and gitsrv, which have PVCs managed by that CSI, that we won't be able to delete the PVCs anymore? Yes, I agree that's a problem, but you'd have the same problem without HNC as well, right? E.g. if you just say "kubectl delete ns csi-driver-nfs docker-registry gitsrv", you'd still have the same race condition, correct?

On Thu, Nov 2, 2023 at 3:25 PM wibed @.***> wrote:

of course heres is an example:

staging├── [s] cert-manager├── [s] csi-driver-nfs├── [s] docker-registry├── [s] gitsrv├── [s] kubernetes-replicator├── [s] metallb-system└── [s] traefik

docker-registry and gitsrv do have a pvc mounted managed by csi-driver-nfs, i must have been overworked at the moment this presented itself to me as a problem, because i should use csi-driver-nfs outside of the staging namespace.

my bad! thank you for your time

— Reply to this email directly, view it on GitHub https://github.com/kubernetes-sigs/hierarchical-namespaces/issues/344#issuecomment-1791419118, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE43PZGNTUPTPU7QYGCDJN3YCPXSBAVCNFSM6AAAAAA62NV7JSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOJRGQYTSMJRHA . You are receiving this because you commented.Message ID: @.***>

wibed commented 10 months ago

indeed. but i wouldnt be exposed to the user friendlyness of a collective namespace such as hnc provides.

adrianludwin commented 10 months ago

Ok - I think it would be pretty challenging for HNC to automatically figure out the relationships. Maybe we could give guidance that dependencies like CSI drivers should be in parent namespaces, and then put finalizers between all objects and those subnamespaces, but that sounds pretty dangerous too. LMK if you have any ideas!

On Thu, Nov 2, 2023 at 3:33 PM wibed @.***> wrote:

indeed. but i wouldnt be exposed to the user friendlyness of a collective namespace such as hnc provides.

— Reply to this email directly, view it on GitHub https://github.com/kubernetes-sigs/hierarchical-namespaces/issues/344#issuecomment-1791429036, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE43PZDBMSL6BPW6ZSYSS43YCPYO5AVCNFSM6AAAAAA62NV7JSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOJRGQZDSMBTGY . You are receiving this because you commented.Message ID: @.***>

wibed commented 10 months ago

i too came to the conclusion that guidance would be the most one could do at this point.

if it demands privileged it should be in a parent namespace or something like that.

adrianludwin commented 10 months ago

sgtm, thanks!

On Thu, Nov 2, 2023 at 3:41 PM wibed @.***> wrote:

i too came to the conclusion that guidance would be the most one could do at this point.

if it demands privileged it should be in a parent namespace or something like that.

— Reply to this email directly, view it on GitHub https://github.com/kubernetes-sigs/hierarchical-namespaces/issues/344#issuecomment-1791439301, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE43PZFKZAREN6AEAF7SCDDYCPZPTAVCNFSM6AAAAAA62NV7JSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOJRGQZTSMZQGE . You are receiving this because you commented.Message ID: @.***>