search

dapr / go-sdk

Dapr SDK for go
Apache License 2.0
446 stars 172 forks source link

panic: error creating default client: error creating connection to '127.0.0.1:50001': context deadline exceeded: context deadline exceeded #259

Closed QQ2287991080 closed 2 years ago

QQ2287991080 commented 2 years ago

i use the https://docs.dapr.io/developing-applications/building-blocks/service-invocation/howto-invoke-discover-services/ go code .

error message: dapr client initializing for: 127.0.0.1:50001 panic: error creating default client: error creating connection to '127.0.0.1:50001': context deadline exceeded: context deadline exceeded

When I use net SDK, it right.

aaronchar commented 2 years ago

I ran into the same issue and went back to v1.2.0 and it seems to have resolved it

weequan93 commented 2 years ago

it is working fine in mac m1 with v1.3.0. Error appear after i run the build (with GOOS=linux and GOARCH=amd64) in linux server

after downgrade to v1.2.0 seem fine.

duffiye commented 2 years ago

maybe can set param for start service "--dapr-grpc-port 50001 "

masters3d commented 2 years ago

I am seeing the same error. I wonder what the issue is and why is not being addressed in next release.

pvr1 commented 2 years ago

I have the same error - Mac m1 on Dapr v1.8.1 using arm64 dapr-cli and rancher desktop 1.4.1 k8s-version:1.24.3 initialised with Dapr init -k

Please advise. Relevant logs etc incl below.

kubectl logs app-bf84fc5c6-4cxtz daprd daprd-container(here all seems fine in the sidecar): 

time="2022-07-20T21:28:32.003694128Z" level=info msg="starting Dapr Runtime -- version 1.8.1 -- commit f2583aa5cf166d83bedc129dfbbd720f100d73d2" app_id=app instance=app-bf84fc5c6-4cxtz scope=dapr.runtime type=log ver=1.8.1
time="2022-07-20T21:28:32.003719044Z" level=info msg="log level set to: debug" app_id=app instance=app-bf84fc5c6-4cxtz scope=dapr.runtime type=log ver=1.8.1
time="2022-07-20T21:28:32.003787044Z" level=info msg="metrics server started on :9090/" app_id=app instance=app-bf84fc5c6-4cxtz scope=dapr.metrics type=log ver=1.8.1
time="2022-07-20T21:28:32.003872419Z" level=info msg="Initializing the operator client (config: pipeline)" app_id=app instance=app-bf84fc5c6-4cxtz scope=dapr.runtime type=log ver=1.8.1

kubectl logs app-bf84fc5c6-4cxtz app app container(relevant part):

dapr client initializing for: 127.0.0.1:50001
...

...
app : [api] 21:29:04 error creating default client: error creating connection to '127.0.0.1:50001': context deadline exceeded: context deadline exceeded
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x697210]

goroutine 1 [running]:
main.main()
    /go/src/app/cmd/app/main.go:144 +0xd70

src:

    pclient, err := dapr.NewClient()
    if err != nil {
        logger.Println(err)
    }
    app.RegisterDaprClient(pclient)
    defer pclient.Close()   ####<---- ROW 144 that triggers error ####

kubectl describe app (relevant part)

  daprd:
    Container ID:  containerd://b73328e0ad877eef7e4092200e1f0df9acbcc2b25f2cd3673416d964ff99fe74
    Image:         docker.io/daprio/daprd:1.8.1
    Image ID:      docker.io/daprio/daprd@sha256:5b283e43badd54cfb8606312da065ee4b7a2fb8c04c0b7259ccd73cddaf3166d
    Ports:         3500/TCP, 50001/TCP, 50002/TCP, 9090/TCP
    Host Ports:    0/TCP, 0/TCP, 0/TCP, 0/TCP
    Args:
      /daprd
      --mode
      kubernetes
      --dapr-http-port
      3500
      --dapr-grpc-port
      50001
      --dapr-internal-grpc-port
      50002
      --dapr-listen-addresses
      [::1],127.0.0.1
      --dapr-public-port
      3501
      --app-port
      8080
      --app-id
      app
      --control-plane-address
      dapr-api.dapr-system.svc.cluster.local:80
      --app-protocol
      grpc
      --placement-host-address
      dapr-placement-server.dapr-system.svc.cluster.local:50005
      --config
      pipeline
      --log-level
      debug
      --app-max-concurrency
      -1
      --sentry-address
      dapr-sentry.dapr-system.svc.cluster.local:80
      --enable-metrics=true
      --metrics-port
      9090
      --dapr-http-max-request-size
      -1
      --dapr-http-read-buffer-size
      -1
      --dapr-graceful-shutdown-seconds
      -1
      --enable-api-logging=false
      --disable-builtin-k8s-secret-store=false
      --enable-mtls
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    2
      Started:      Wed, 20 Jul 2022 23:46:29 +0200
      Finished:     Wed, 20 Jul 2022 23:46:46 +0200

go.sum (relevant): 

github.com/dapr/dapr v1.8.1 h1:vRnPmIfaKmk9gbRP7IQF1beZfRK37D26iMtNOqhxl04=
github.com/dapr/dapr v1.8.1/go.mod h1:wg91Q7q3OhZ3I+HLoZGCieAGygoYrUcGsdK6N4wMHVY=
github.com/dapr/go-sdk v1.5.0 h1:OVkrupquJEOL1qRtwKcMVrFKYhw4UJQvgOJNduo2VxE=
github.com/dapr/go-sdk v1.5.0/go.mod h1:Cvz3taCVu22WCNEUbc9/szvG/yJxWPAV4dcaG+zDWA4=

go.mod:

module app

go 1.18

require (
        github.com/dapr/go-sdk v1.5.0
        github.com/grpc-ecosystem/go-grpc-middleware v1.3.0
        goa.design/goa/v3 v3.7.12
        google.golang.org/grpc v1.48.0
        google.golang.org/protobuf v1.28.0
)

require go.mongodb.org/mongo-driver v1.10.0

require (
        github.com/hashicorp/errwrap v1.1.0 // indirect
        github.com/hashicorp/go-multierror v1.1.1 // indirect
        github.com/montanaflynn/stats v0.6.6 // indirect
        github.com/myesui/uuid v1.0.0 // indirect
        gopkg.in/stretchr/testify.v1 v1.2.2 // indirect
)

require (
        github.com/dapr/dapr v1.8.1 // indirect
        github.com/dimfeld/httppath v0.0.0-20170720192232-ee938bf73598 // indirect
        github.com/dimfeld/httptreemux/v5 v5.4.0 // indirect
        github.com/golang/protobuf v1.5.2 // indirect
        github.com/golang/snappy v0.0.4 // indirect
        github.com/google/uuid v1.3.0 // indirect
        github.com/gopherjs/gopherjs v0.0.0-20200217142428-fce0ec30dd00 // indirect
        github.com/gorilla/websocket v1.5.0 // indirect
        github.com/jtolds/gls v4.20.0+incompatible // indirect
        github.com/klauspost/compress v1.15.8 // indirect
        github.com/manveru/faker v0.0.0-20171103152722-9fbc68a78c4d // indirect
        github.com/pkg/errors v0.9.1 // indirect
        github.com/sergi/go-diff v1.2.0 // indirect
        github.com/smartystreets/assertions v1.1.0 // indirect
        github.com/twinj/uuid v1.0.0
        github.com/xdg-go/pbkdf2 v1.0.0 // indirect

dapr-system namespace:

kubectl get po -n dapr-system
NAME                                     READY   STATUS    RESTARTS   AGE
dapr-dashboard-85ddc7bbd6-6dkp5          1/1     Running   0          44m
dapr-operator-6459998c48-ls7tz           1/1     Running   0          44m
dapr-sentry-6477f676c-s5fgc              1/1     Running   0          44m
dapr-placement-server-0                  1/1     Running   0          44m
dapr-sidecar-injector-5dd984d956-ktlfz   1/1     Running   0          44m

After looking into the code of Dapr/go-sdk I saw that it fails in https://github.com/dapr/go-sdk/blob/v1.5.0/client/client.go in the 

// NewClientWithAddress instantiates Dapr using specific address (including port).
func NewClientWithAddress(address string) (client Client, err error) {
    if address == "" {
        return nil, errors.New("nil address")
    }
    logger.Printf("dapr client initializing for: %s", address)

    ctx, ctxCancel := context.WithTimeout(context.Background(), 1*time.Second)
    conn, err := grpc.DialContext(
        ctx,
        address,
        grpc.WithTransportCredentials(insecure.NewCredentials()),
        grpc.WithBlock(),
    )
    if err != nil {
        ctxCancel()
        return nil, errors.Wrapf(err, "error creating connection to '%s': %v", address, err)
    }
    if hasToken := os.Getenv(apiTokenEnvVarName); hasToken != "" {
        logger.Println("client uses API token")
    }

    return newClientWithConnectionAndCancelFunc(conn, ctxCancel), nil
}

this I deduced from the error message - 

[api] 21:29:04 error creating default client: error creating connection to '127.0.0.1:50001': context deadline exceeded: context deadline exceeded

after looking into the context I noticed - 

ctx, ctxCancel := context.WithTimeout(context.Background(), 1*time.Second)

Is it a too short with 1*time.Second? Esp for dev k8s setups? I checked git blame for the context timeout line and found

https://github.com/dapr/go-sdk/commit/5d3a4ad3f230b2421f998f9f9975cca5cb1e8b01

that commit was at the time the above complaints. is it a coincidence?

UPDATE - fixed everything by altering 1) ctx, ctxCancel := context.WithTimeout(context.Background(), 1*time.Second)
TO
ctx, ctxCancel := context.WithTimeout(context.Background(), 5*time.Second) 2) Adding

        dapr.io/sidecar-liveness-probe-delay-seconds: "10"
        dapr.io/sidecar-liveness-probe-timeout-seconds: "10"
        dapr.io/sidecar-liveness-probe-threshold: "10"
        dapr.io/sidecar-readiness-probe-delay-seconds: "10"
        dapr.io/sidecar-readiness-probe-timeout-seconds: "10"

to the yaml for deploying my micro service. Seems like Dapr is getting heavy? And a DEV/k8s setup on a laptop cannot really cope with production settings.

yaron2 commented 2 years ago

Closed via #310