Silencing `Connection Refused` logs

[MoreNaruto]

• Which Faktory package and version? Faktory Enterprise 1.6.0
• Which Faktory worker package and version? "github.com/contribsys/faktory_worker_go" 1.6.0
• Please include any relevant worker configuration
• Please include any relevant error messages or stacktraces

Are you using an old version? Have you checked the changelogs to see if your issue has been fixed in a later version?

https://github.com/contribsys/faktory/blob/master/Changes.md https://github.com/contribsys/faktory/blob/master/Pro-Changes.md https://github.com/contribsys/faktory/blob/master/Ent-Changes.md

So whenever our faktory worker is down, the connection will constantly return this error. Is there a way to silence this particular log after the first failure? Or is there a way to have exponential retry logic to try to reconnect?

[mperham]

This issue belongs in the contribsys/faktory_worker_go repo, as it's a worker issue. The code there is all open source. I'd welcome a PR if you have a specific fix in mind.

[MoreNaruto]

So in this file: https://github.com/contribsys/faktory_worker_go/blob/main/runner.go

I'm going to replace L83-102 with:

//Implement exponential backoff for creating a process
    sleep := 1
    for i := 0; i < 10000; i++ {
        if mgr.state != "" {
            return
        }

        // check for shutdown
        select {
        case <-mgr.done:
            return
        default:
        }

        err := processOne(mgr)
        if err != nil {
            mgr.Logger.Debug(err)
            if _, ok := err.(*NoHandlerError); !ok {
                time.Sleep(1 * time.Second)
            }
        }
        time.Sleep(time.Duration(sleep) * time.Second)
        sleep *= 2
    }

There's a repo: github.com/cenkalti/backoff that can also do it, but I'm not sure what the rules are for incorporating new dependencies into your repo, so here's the boilerplate version.

I can't make a PR since I'm not a contributor. So can you add me as a contributor or is there another way you want me to create the PR?

[mperham]

Creating a PR is pretty simple with GitHub. You Fork the repo, create a new branch, make your changes, test them, commit them, push them to your fork and then come back to github UI where it will allow you to create a PR based on your fork’s branch.

On Fri, May 6, 2022 at 15:02 Thomas Morris @.***> wrote:

So in this file: https://github.com/contribsys/faktory_worker_go/blob/main/runner.go

I'm going to replace L83-102 with:

//Implement exponential backoff for creating a process sleep := 1 for i := 0; i < 10000; i++ { if mgr.state != "" { return }

  // check for shutdown
  select {
  case <-mgr.done:
      return
  default:
  }

  err := processOne(mgr)
  if err != nil {
      mgr.Logger.Debug(err)
      if _, ok := err.(*NoHandlerError); !ok {
          time.Sleep(1 * time.Second)
      }
  }
  time.Sleep(time.Duration(sleep) * time.Second)
  sleep *= 2

}

There's a repo: github.com/cenkalti/backoff that can also do it, but I'm not sure what the rules are for incorporating new dependencies into your repo, so here's the boilerplate version.

I can't make a PR since I'm not a contributor. So can you add me as a contributor or is there another way you want me to create the PR?

— Reply to this email directly, view it on GitHub https://github.com/contribsys/faktory/issues/404#issuecomment-1120041991, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAAWX24JFTBPODFY2EWIELVIWJILANCNFSM5VJI2NMQ . You are receiving this because you commented.Message ID: @.***>