Attempting to stop a server that was evicted or manually deleted should work

[consideRatio]

I'm not confident, but I think that if a user server is evicted from a node (memory pressure, forceful drain operation, etc) or manually deleted with kubectl delete pod, a user pressing "stop my server" will get errors etc. and fail to stop the server which is required to start it again.

I figure we should verify that my understanding is correct that KubeSpawner isn't gracefully stopping a evicted server pod and straight up deleted pod, but instead surfaces hard to parse errors and blocks the jupyterhub user from restarting the server. And then, enabling KubeSpawner to gracefully stop user servers and restart them if this happens.

[consideRatio]

I think kubectl delete may delete the Pod object, while an eviction may retain it in a terminated state or some kind. So to test how kubespawner handles evicted pods, I think the best way to go about it is to make an API request to evict a pod according to these k8s official docs

[dolfinus]

Well, _make_delete_pod_request already handles missing pod: https://github.com/jupyterhub/kubespawner/blob/5c6801b9d87508e2435b2dd11da8b89040a72ef6/kubespawner/spawner.py#L2867

So kubectl delete should not be an issue.

[consideRatio]

@dolfinus I think that 404 check isn't enough, a pod that has been evicted can be found still in a non-running evicted state I think. I'm still not confident on the details, and have not recently reproduced this.

[dolfinus]

Still don't get why evicted pod cannot be deleted. It's just the same case as pod is not started yet.