showjason
commented
2 months ago /assign
Open bpinske opened 3 months ago
showjason
commented
2 months ago /assign
showjason
commented
2 months ago @bpinske from the Kubernetes source code, we can see the endpoints must not be empty, so your endpointslice node-local-dns-rfnd6 is incorrect.
bpinske
commented
2 months ago maybe this is my ignorance showing, but how does that say endpoints cannot be null?
I have many endpointslices it seems which have null endpoints. Including very old stuff like Helm Tiller which I haven't used in years. So it definitely happens regularly and as a normal thing. I don't think the python client library should fail to handle this when the Go library does handle it properly.
Further, I ended up just writing my tool in golang to unblock mysel. I do have the desired and expected behaviour when using Go.
Here is a simple Go version which does have the expected behaviour and does handle null endpoints.
import (
"context"
"fmt"
"os"
"path/filepath"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/rest"
"k8s.io/client-go/tools/clientcmd"
"k8s.io/client-go/util/homedir"
_ "k8s.io/client-go/plugin/pkg/client/auth/oidc"
)
const (
kubeconfigEnvVar = "KUBERNETES_SERVICE_HOST"
)
func main() {
config, err := getKubeConfig()
if err != nil {
fmt.Printf("Failed to get Kubernetes config: %s\n", err)
os.Exit(1)
}
clientset, err := kubernetes.NewForConfig(config)
if err != nil {
fmt.Printf("Failed to create Kubernetes client: %s\n", err)
os.Exit(1)
}
printEndpointSlices(clientset)
}
func getKubeConfig() (*rest.Config, error) {
if _, exists := os.LookupEnv(kubeconfigEnvVar); exists {
fmt.Println("Running in cluster detected")
return rest.InClusterConfig()
}
fmt.Println("Running outside of cluster detected")
kubeconfigPath := filepath.Join(homedir.HomeDir(), ".kube", "config")
return clientcmd.BuildConfigFromFlags("", kubeconfigPath)
}
func printEndpointSlices(clientset *kubernetes.Clientset) {
fmt.Println("Fetching EndpointSlices...")
slices, err := clientset.DiscoveryV1().EndpointSlices("").List(context.Background(), metav1.ListOptions{})
if err != nil {
fmt.Printf("Error fetching EndpointSlices: %s\n", err)
os.Exit(1)
}
for _, slice := range slices.Items {
fmt.Printf("Namespace: %s, Name: %s, Endpoints:\n", slice.Namespace, slice.Name)
for _, endpoint := range slice.Endpoints {
fmt.Printf(" - Addresses: %v\n", endpoint.Addresses)
}
}
}outputs
Namespace: default, Name:populatedendpointslice-8qjxn, Endpoints:
- Addresses: [10.2.110.161]
- Addresses: [10.2.206.229]
Namespace: default, Name: emptyendpointslice-7pwhd, Endpoints:Hi @bpinske , sorry, it's my stupid mistake, the source code means field endpoints is required, but not means its value must not be empty.
I also tried to create an endpointslice with empty endpoints via kubectl and succeeded.
Let me open a PR to fix this bug
showjason
commented
2 months ago Hi @roycaihw , this endpoints is not validated properly, it's generated by OpenAPI Generator automatically and not allowed to edit manually, hence I check the Kubernetes source code and find that there are specific validation functions() validating the endpointslices and endpoints in validation.go,but they are not generated by OpenAPI Generator as expected. If my understanding is correct, do you think this bug will be fixed in the coming release?
showjason
commented
1 month ago Issue https://github.com/kubernetes-client/python/issues/1662 can resolve this problem by disabling the client side validation
anupamdialpad
commented
1 month ago Hi @showjason,
I am using it inside a watcher can you tell me where to add client_side_validation. I tried the below code but it still fails with "Invalid value for endpoints, must not be None"
from kubernetes import watch, config, client
apiserver = client.DiscoveryV1Api()
apiserver.api_client.client_side_validation = False
watcher = watch.Watch()
for event in watcher.stream(apiserver.list_namespaced_endpoint_slice, 'default', label_selector=label_selector, timeout_seconds=10):
print(event)Hi @anupamdialpad,
About the functions start with list_, they will deserialize the response data which is due to _preload_content set to True by default, this error occurs during deserialization.
Let's take list_namespaced_endpoint_slice as an example:
By default, function __deserialize will be called to deserialize the object V1EndpointSlice, at this time, you can see this line, the Configuration is reset by initializing with default parameters which leads to the client_side_validation is reset to True.
After diving into the code, I find this line is wired, the comment is Disable the client side validation, but actually the value is True which means it enables the client side validation. CC @roycaihw
The way to address this issue is to set the param _preload_content of functions list_xxx to False to bypass the deserialization. After this, you can get the response data of byte type, then serialize the data to Json or other type you expect by yourself.
from kubernetes import client, config
config.load_kube_config() # For local development
discovery_api = client.DiscoveryV1Api()
endpoint_slices = discovery_api.list_namespaced_endpoint_slice(namespace="default", _preload_content=False)
print(endpoint_slices.data)
What happened (please include outputs or screenshots):
Cannot list endpoint slices using the kubernetes python library in a cluster where one of the endpoint slices contains a null list of endpoints.
Fails with the following stack trace.
What you expected to happen:
Expected a list of endpointslices to be returned.
How to reproduce it (as minimally and precisely as possible):
Create an endpointslice with endpoints as null. See bottom of issue for example of null endpoints.
Execute this code on any kubernetes cluster.
Anything else we need to know?:
Noteworthy that if I debug this a bit, I do see that I am receiving the EndpointSliceList correctly from the apiserver, but the client is struggling to do...something with it.
==== After further investigation, it appears related to endpointslices with null endpoints for example. If I debug down, I find that it's failing to deserialize on this particular endpointslice in my cluster.