amenzhinsky / iothub

Azure IoT Hub SDK for Golang
MIT License
51 stars 57 forks source link

Fatal error link detached due to IdleTimerExpired #50

Open lohmanndouglas opened 3 years ago

lohmanndouglas commented 3 years ago

Version:

github.com/amenzhinsky/iothub v0.7.0

Error:

Link detached, reason: *Error{Condition: amqp:link:detach-forced, Description: The link 'G2:...' is force detached. Code: ServerError. Details: AmqpEventHubConsumer.IdleTimerExpired: Idle timeout: 00:30:00.

I am getting this error message and restarting the pod Kubernetes container frequently. I had this code working for a long time and it did not happen so frequently. The first time, I thought that it was idle, but I checked the last message received was one minute before and not 30 minutes, maybe I misunderstand the Azure concept of idle in the Iothub bus and be observing the wrong parameters.

My contantainer log is:

... {"time":"2021-05-01T06:17:58.627919145Z", "message received"} {"time":"2021-05-01T06:17:58.633331806Z", "message received"} {"time":"2021-05-01T06:17:58.638524857Z", "message received"} {"time":"2021-05-01T06:18:02.227702684Z","level":"FATAL","prefix":"-","file":"main.go","line":"47","message":"link detached, reason: *Error{Condition: amqp:link:detach-forced, Description: The link 'G2:omited' is force detached. Code: ServerError. Details: AmqpEventHubConsumer.IdleTimerExpired: Idle timeout: 00:30:00.}

My golang code is:

parentCtx, _ := context.WithCancel(context.Background())
log.Fatal(iotHub.SubscribeEvents(parentCtx, func(msg *iotservice.Event) error {
    err = json.Unmarshal(msg.Payload, &identifier)
    log.Info(" identifier :", identifier)
    if err == nil {
                 HandleID(identifier)
    } else {
        log.Error("could not marshing identifier: ", err)
    }
       return nil
}

I am considering doing something like the following draft, but a little afraid of losing messages, now if there is an error I see due to the container reinitialization and fix it as soon as possible to do not lose messages.

draft:

for {
       err = iotHub.SubscribeEvents(parentCtx, HandleFuncHere)
       if err != nil {
        if err == context.DeadlineExceeded { // IdleTimerExpired?
                     log.Errorf("iothub idle timeout")
        } else {
            log.Fatal(err) // another error
        }
       }
    continue
}

This is a reasonable solution? Did someone experience something similar?

I appreciate any help that someone provides to workaround or understand better this issue, thanks :)