amenzhinsky / iothub

Azure IoT Hub SDK for Golang
MIT License
51 stars 57 forks source link

Connection drops after a while, how to refresh it #38

Open tappoz opened 3 years ago

tappoz commented 3 years ago

I'm creating a cloud-to-device link via AMQP with:

ihQueueClient, err := iotservice.NewFromConnectionString(iotHubConnStr)

I can use this successfully for 1 hour or so, but then after that I start seeing:

*Error{Condition: amqp:unauthorized-access, Description: Token or Certificate is invalid., Info: map[com.microsoft:is-filtered:true com.microsoft:tracking-id:<<<TRACKING_ID>>>-G:6-TimeStamp:02/10/2021 16:05:08]}

Looks like the AMQP connection is being dropped due to inactivity or just because 1 hour is passed after it has been created. Is there any way I can avoid this? How can I specify to automatically refresh or reconnect to the IoT Hub?

I can see in here https://github.com/amenzhinsky/iothub/blob/e718b48d126cf72b9696022028762ccd1d9ee1fe/iotservice/client.go#L44

there's this TLS option, but it's not exactly related to timeouts, that's more about extra security checks when using TLS:

// WithTLSConfig sets TLS config that's used by REST HTTP and AMQP clients.
func WithTLSConfig(config *tls.Config) ClientOption {
    return func(c *Client) {
        c.tls = config
    }
}
amenzhinsky commented 3 years ago

If I recall correctly we had this problem before, the client should maintain token actuality in the background:

https://github.com/amenzhinsky/iothub/blob/master/iotservice/client.go#L147

tappoz commented 3 years ago

There is this stack of calls:

That is not using consistently a Go context - it is recreating in a few different places context.Background(). It would be useful to be able to create a Go context with a timeout so we can invalidate a session. Also these are mostly unexposed so I can not call them from outside the Go package where they are written.

This does not allow to refresh the IoT Hub token and the effect is that having a long running process that is supposed to be constantly connected to the IoT Hub to send "could-to-device" messages does not work. This long running process stops working after around 1 hour from when it was started.

I ended up working around this issue with:

func (ms *myStruct) IotHubRefreshConnection(iotHubConnStr string) {
    if ms.ihsClient != nil {
        log.Info.Println("Closing the IoT Hub connection...")
        ms.ihsClient.Close()
    }
    log.Printf("Connecting to the IoT Hub with: %v", iotHubConnStr)
    ihQueueClient, err := iotservice.NewFromConnectionString(
        iotHubConnStr,
        iotservice.WithLogger(
            logger.New(
                logger.LevelDebug,
                func(lvl logger.Level, s string) {
                    log.Println(fmt.Sprint("*IH-CLIENT* ", lvl.String(), " ", s))
                },
            ),
        ),
    )
    if err != nil {
        log.Error.Printf("Cannot establish the IoT Hub connection: %+v", err)
        panic(err)
    }
    ms.ihsClient = ihQueueClient
}

func syncRefreshSchedule(ms *myStruct, iotHubConnStr string) {
    for {
        log.Info.Println("Wating to refresh the Iot Hub connection...")
        select {
        case <-time.After(10 * time.Minute):
            log.Info.Println("Recurrent time to refresh the IoT Hub connection...")
            ms.IotHubRefreshConnection(iotHubConnStr)
        }
    }
}

Then in my constructor of myStruct I do this:

ms.IotHubRefreshConnection(iotHubConnStr)
go syncRefreshSchedule(&ms, iotHubConnStr)

This way I make sure there is always a working IoT Hub session. Even after 1 hour.

amenzhinsky commented 3 years ago

Can you try the latest master, there was simply a defer error that closed amqp session before it tired to renew token.