amir / raidman

Go Riemann client
The Unlicense
68 stars 25 forks source link

Reconnect #4

Open ProTip opened 10 years ago

ProTip commented 10 years ago

Hello,

What do you think about handling reconnects? Currently there is nothing exposing whether or not the client is still connected nor is there a "Connect" method to attempt a reconnection.

I'm also curious about having the client handle broken connections itself.. I had to do the same stuff with the graphite client (wrap a bunch of broken connection handling code around it).

blalor commented 9 years ago

It's not particularly elegant, but here's what I use:

package main

import (
    log "github.com/Sirupsen/logrus"

    "time"
    "fmt"

    "github.com/amir/raidman"
)

type ReconnectingRiemannClient struct {
    Proto   string
    Host    string
    Port    int
    Timeout time.Duration

    client  *raidman.Client
}

func (self *ReconnectingRiemannClient) SendMulti(evts []*raidman.Event) (err error) {
    if self.client == nil {
        riemannAddr := fmt.Sprintf("%s:%d", self.Host, self.Port)

        log.Infof("connecting to Riemann at %s via %s", riemannAddr, self.Proto)
        self.client, err = raidman.DialWithTimeout(self.Proto, riemannAddr, self.Timeout)

        if err != nil {
            log.Errorf("error connecting to Riemann: %v", err)
            return
        }
    }

    if self.client != nil {
        err = self.client.SendMulti(evts)

        if err != nil {
            self.client.Close()
            self.client = nil
        }
    }

    return
}
amir commented 9 years ago

Sorry for the long silence. I've read quite a few go projects to see how they handle reconnects since the issue has been created. I'm yet to see one uniform, idiomatic solution to the problem. @blalor's solution is indeed a good one from a library user point of view and we might be able to avail users of this feature via a configuration parameter, perhaps? What do you think?

blalor commented 9 years ago

I'm wondering if it might be better to keep this as a separate implementation that uses a common interface to describe the methods. This is working well for me, but I'm a little uncomfortable having the library wrap itself; seems like it could make maintenance more complex. But as a separate type, its behavior and relationship are more understandable, in my opinion.

dhruvbansal commented 8 years ago

+1 for bundling some kind of reconnect functionality into the client. Can't speak to @blalor particular implemenation...IANAGP (IANA Go Programmer...).

Metrics are what we rely on when things break so it's nice when our collection tools are robust against intermittent network errors or (say) a restart of the Riemann server (which is how I discovered this bug & issue...).