openebs / mayastor

Dynamically provision Stateful Persistent Replicated Cluster-wide Fabric Volumes & Filesystems for Kubernetes that is provisioned from an optimized NVME SPDK backend data storage stack.
Apache License 2.0
669 stars 101 forks source link

kubectl mayastor : Failed to initialise the REST client. Error HyperError: error trying to connect: dns error: failed to lookup address information: Name does not resolve #1617

Open todeb opened 3 months ago

todeb commented 3 months ago

Are you reporting an issue with existing content? Please describe the error or omission, or your suggestion for improvement. Please include a link the affected page(s) Doc: https://mayastor.gitbook.io/introduction/advanced-operations/kubectl-plugin

When using these commands as it is: Failed to initialise the REST client. Error HyperError: error trying to connect: dns error: failed to lookup address information: Name does not resolve

Are you proposing new content, or a change to the existing documentation layout or structure? Please describe your proposal.

Please add information that to make it work it is needed to do port forward and use it with -r. eg:

kubectl port-forward svc/mayastor-api-rest -n mayastor 8080
kubectl-mayastor get nodes -r https://localhost:8080

If other config is required to not need port-forward, it also can be added to doc.

todeb commented 3 months ago

Don't know why but on my other pc it started working out of the box without -r . Need to verify on my initial one.

tiagolobocastro commented 3 months ago

You shouldn't need to port-forward and use -r as the plugin kinda does that by itself. But it only started doing this from v2 onwards so please check the plugin version.

todeb commented 2 months ago

I had checked a little bit more. I have now the newest version of plugin 1.26. Although the issue still persists. For more information I'm using wsl with ububntu22.

When my resolv.conf is set which is default to: 172.24.16.1 which points to Ethernet adapter vEthernet (WSL (Hyper-V firewall)) Then mayastor out dns error.

If i change dns server in resolve to my dns server then it is working without dns error. Everything what i have so far except mayastor are accepting WSL default resolv.

Is there any way to have this working? Is it possible to enable debug on client and check what the problem is? how? What exact name mayastor plugin is looking for?

wsl --version
WSL version: 2.1.5.0
Kernel version: 5.15.146.1-2
WSLg version: 1.0.60
MSRDC version: 1.2.5105
Direct3D version: 1.611.1-81528511
DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows version: 10.0.22621.3447
tiagolobocastro commented 2 months ago

Great, thanks for debugging that!

Could you try running with logs, like so:

RUST_LOG=trace kubectl-mayastor get nodes

Although not sure if that will add anything useful, let's see.

todeb commented 2 months ago

Working response:

RUST_LOG=trace kubectl-mayastor get nodes
  2024-04-22T09:27:21.634710Z DEBUG kube_client::client::builder: requesting
    at /sources/kube-client-0.85.0-98989b6e1f27695afe22aa29c94136fa06be5e8d28b91222e6dfbe5a460c803f/src/client/builder.rs:127
    in kube_client::client::builder::HTTP with http.method: GET, http.url: https://myfqdn/k8s/clusters/c-1234/api/v1/namespaces/mayastor/services?&labelSelector=app%3Dapi-rest, otel.name: "list", otel.kind: "client"

  2024-04-22T09:27:24.576593Z  INFO kube_forward::http_forward: generated kube-api, uri: /api/v1/namespaces/mayastor/services/http:mayastor-api-rest:http/proxy
    at dependencies/control-plane/k8s/forward/src/http_forward.rs:54

  2024-04-22T09:27:24.621326Z TRACE openapi::clients::tower::configuration: started GET /api/v1/namespaces/mayastor/services/http:mayastor-api-rest:http/proxy/v0/nodes
    at dependencies/control-plane/openapi/src/clients/tower/configuration.rs:218
    in openapi::clients::tower::configuration::HTTP with http.method: GET, http.url: /api/v1/namespaces/mayastor/services/http:mayastor-api-rest:http/proxy/v0/nodes, otel.name: GET /api/v1/namespaces/mayastor/services/http:mayastor-api-rest:http/proxy/v0/nodes, otel.kind: "client"

  2024-04-22T09:27:24.622390Z DEBUG kube_client::client::builder: requesting
    at /sources/kube-client-0.85.0-98989b6e1f27695afe22aa29c94136fa06be5e8d28b91222e6dfbe5a460c803f/src/client/builder.rs:127
    in kube_client::client::builder::HTTP with http.method: GET, http.url: https://myfqdn/k8s/clusters/c-1234/api/v1/namespaces/mayastor/services/http:mayastor-api-rest:http/proxy/v0/nodes, otel.name: "HTTP", otel.kind: "client"
    in openapi::clients::tower::configuration::HTTP with http.method: GET, http.url: /api/v1/namespaces/mayastor/services/http:mayastor-api-rest:http/proxy/v0/nodes, otel.name: GET /api/v1/namespaces/mayastor/services/http:mayastor-api-rest:http/proxy/v0/nodes, otel.kind: "client"

Not working response:

RUST_LOG=trace kubectl-mayastor get nodes
  2024-04-22T09:26:07.310313Z DEBUG kube_client::client::builder: requesting
    at /sources/kube-client-0.85.0-98989b6e1f27695afe22aa29c94136fa06be5e8d28b91222e6dfbe5a460c803f/src/client/builder.rs:127
    in kube_client::client::builder::HTTP with http.method: GET, http.url: https://myfqdn/k8s/clusters/c-1234/api/v1/namespaces/mayastor/services?&labelSelector=app%3Dapi-rest, otel.name: "list", otel.kind: "client"

  2024-04-22T09:26:07.438229Z ERROR kube_client::client::builder: failed with error error trying to connect: dns error: failed to lookup address information: Name does not resolve
    at /sources/kube-client-0.85.0-98989b6e1f27695afe22aa29c94136fa06be5e8d28b91222e6dfbe5a460c803f/src/client/builder.rs:154
    in kube_client::client::builder::HTTP with http.method: GET, http.url: https://myfqdn/k8s/clusters/c-1234/api/v1/namespaces/mayastor/services?&labelSelector=app%3Dapi-rest, otel.name: "list", otel.kind: "client", otel.status_code: "ERROR"

Failed to initialise the REST client. Error HyperError: error trying to connect: dns error: failed to lookup address information: Name does not resolve