search

openebs / mayastor

Dynamically provision Stateful Persistent Replicated Cluster-wide Fabric Volumes & Filesystems for Kubernetes that is provisioned from an optimized NVME SPDK backend data storage stack.
Apache License 2.0
754 stars 109 forks source link

kubectl mayastor : Failed to initialise the REST client. Error HyperError: error trying to connect: dns error: failed to lookup address information: Name does not resolve #1617

Closed todeb closed 1 month ago

todeb commented 8 months ago

Are you reporting an issue with existing content? Please describe the error or omission, or your suggestion for improvement. Please include a link the affected page(s) Doc: https://mayastor.gitbook.io/introduction/advanced-operations/kubectl-plugin

When using these commands as it is: Failed to initialise the REST client. Error HyperError: error trying to connect: dns error: failed to lookup address information: Name does not resolve

Are you proposing new content, or a change to the existing documentation layout or structure? Please describe your proposal.

Please add information that to make it work it is needed to do port forward and use it with -r. eg:

kubectl port-forward svc/mayastor-api-rest -n mayastor 8080
kubectl-mayastor get nodes -r https://localhost:8080

If other config is required to not need port-forward, it also can be added to doc.

todeb commented 7 months ago

Don't know why but on my other pc it started working out of the box without -r . Need to verify on my initial one.

tiagolobocastro commented 7 months ago

You shouldn't need to port-forward and use -r as the plugin kinda does that by itself. But it only started doing this from v2 onwards so please check the plugin version.

todeb commented 7 months ago

I had checked a little bit more. I have now the newest version of plugin 1.26. Although the issue still persists. For more information I'm using wsl with ububntu22.

When my resolv.conf is set which is default to: 172.24.16.1 which points to Ethernet adapter vEthernet (WSL (Hyper-V firewall)) Then mayastor out dns error.

If i change dns server in resolve to my dns server then it is working without dns error. Everything what i have so far except mayastor are accepting WSL default resolv.

Is there any way to have this working? Is it possible to enable debug on client and check what the problem is? how? What exact name mayastor plugin is looking for?

wsl --version
WSL version: 2.1.5.0
Kernel version: 5.15.146.1-2
WSLg version: 1.0.60
MSRDC version: 1.2.5105
Direct3D version: 1.611.1-81528511
DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows version: 10.0.22621.3447
tiagolobocastro commented 7 months ago

Great, thanks for debugging that!

Could you try running with logs, like so:

RUST_LOG=trace kubectl-mayastor get nodes

Although not sure if that will add anything useful, let's see.

todeb commented 7 months ago

Working response:

RUST_LOG=trace kubectl-mayastor get nodes
  2024-04-22T09:27:21.634710Z DEBUG kube_client::client::builder: requesting
    at /sources/kube-client-0.85.0-98989b6e1f27695afe22aa29c94136fa06be5e8d28b91222e6dfbe5a460c803f/src/client/builder.rs:127
    in kube_client::client::builder::HTTP with http.method: GET, http.url: https://myfqdn/k8s/clusters/c-1234/api/v1/namespaces/mayastor/services?&labelSelector=app%3Dapi-rest, otel.name: "list", otel.kind: "client"

  2024-04-22T09:27:24.576593Z  INFO kube_forward::http_forward: generated kube-api, uri: /api/v1/namespaces/mayastor/services/http:mayastor-api-rest:http/proxy
    at dependencies/control-plane/k8s/forward/src/http_forward.rs:54

  2024-04-22T09:27:24.621326Z TRACE openapi::clients::tower::configuration: started GET /api/v1/namespaces/mayastor/services/http:mayastor-api-rest:http/proxy/v0/nodes
    at dependencies/control-plane/openapi/src/clients/tower/configuration.rs:218
    in openapi::clients::tower::configuration::HTTP with http.method: GET, http.url: /api/v1/namespaces/mayastor/services/http:mayastor-api-rest:http/proxy/v0/nodes, otel.name: GET /api/v1/namespaces/mayastor/services/http:mayastor-api-rest:http/proxy/v0/nodes, otel.kind: "client"

  2024-04-22T09:27:24.622390Z DEBUG kube_client::client::builder: requesting
    at /sources/kube-client-0.85.0-98989b6e1f27695afe22aa29c94136fa06be5e8d28b91222e6dfbe5a460c803f/src/client/builder.rs:127
    in kube_client::client::builder::HTTP with http.method: GET, http.url: https://myfqdn/k8s/clusters/c-1234/api/v1/namespaces/mayastor/services/http:mayastor-api-rest:http/proxy/v0/nodes, otel.name: "HTTP", otel.kind: "client"
    in openapi::clients::tower::configuration::HTTP with http.method: GET, http.url: /api/v1/namespaces/mayastor/services/http:mayastor-api-rest:http/proxy/v0/nodes, otel.name: GET /api/v1/namespaces/mayastor/services/http:mayastor-api-rest:http/proxy/v0/nodes, otel.kind: "client"

Not working response:

RUST_LOG=trace kubectl-mayastor get nodes
  2024-04-22T09:26:07.310313Z DEBUG kube_client::client::builder: requesting
    at /sources/kube-client-0.85.0-98989b6e1f27695afe22aa29c94136fa06be5e8d28b91222e6dfbe5a460c803f/src/client/builder.rs:127
    in kube_client::client::builder::HTTP with http.method: GET, http.url: https://myfqdn/k8s/clusters/c-1234/api/v1/namespaces/mayastor/services?&labelSelector=app%3Dapi-rest, otel.name: "list", otel.kind: "client"

  2024-04-22T09:26:07.438229Z ERROR kube_client::client::builder: failed with error error trying to connect: dns error: failed to lookup address information: Name does not resolve
    at /sources/kube-client-0.85.0-98989b6e1f27695afe22aa29c94136fa06be5e8d28b91222e6dfbe5a460c803f/src/client/builder.rs:154
    in kube_client::client::builder::HTTP with http.method: GET, http.url: https://myfqdn/k8s/clusters/c-1234/api/v1/namespaces/mayastor/services?&labelSelector=app%3Dapi-rest, otel.name: "list", otel.kind: "client", otel.status_code: "ERROR"

Failed to initialise the REST client. Error HyperError: error trying to connect: dns error: failed to lookup address information: Name does not resolve
tiagolobocastro commented 1 month ago

@todeb

When my resolv.conf is set which is default to: 172.24.16.1 which points to Ethernet adapter vEthernet (WSL (Hyper-V firewall))

In this case does kubectl by itself work?

fara-tode commented 1 month ago

yes kubectl was working at that time without issue.

Now all is working without issue so ticket can be closed. But lot of changed during months, so all specs are different:

Kubectl Plugin (kubectl-mayastor) revision d0a6618f4898 (v2.7.0+0)

WSL version: 2.3.24.0
Kernel version: 5.15.153.1-2
WSLg version: 1.0.65
MSRDC version: 1.2.5620
Direct3D version: 1.611.1-81528511
DXCore version: 10.0.26100.1-240331-1435.ge-release
Windows version: 10.0.22621.4317

also Im using now new chart and have everything in openebs not mayastor namespace.

So not sure what was the issue, but now it is working without need to do any port forwards or workaround with resolvers.

tiagolobocastro commented 1 month ago

Interesting, I don't think we've specifically fixed anything related to this... Alright, let's close this for now then, and hope it's somehow resolved or if reopen we can take another look. Thank you