contribsys / faktory

Language-agnostic persistent background job server
https://contribsys.com/faktory/
Other
5.78k stars 230 forks source link

Faktory sequential jobs > 2 documentation question #488

Open profsmallpine opened 2 weeks ago

profsmallpine commented 2 weeks ago

I am wondering if you can have multiple sequential jobs. Given that a success callback requires a job and not a batch, how would you handle sequences > 2?

    err := s.Pool.With(func(c *client.Client) error {
        job3 := client.NewJob("job-3", arg3)
        job2 := client.NewJob("job-2", arg2)
        job1 := client.NewJob("job-1", arg1)

        b := client.NewBatch(c)
        b.Success = job2
        err := b.Jobs(func() error {
            return b.Push(job1)
        })

        return err
    })

    return err

I don't see how I would tell job3 to execute after job2 completes. After reading https://github.com/contribsys/faktory/wiki/Ent-Batches, I assumed this was possible but can't seem to figure out the right syntax (if it's possible). Any guidance here?

mperham commented 2 weeks ago

That's the purpose of nested batches. You have a parent batch A with child batch B full of jobs. When those jobs are done, B's success callback fires; the callback opens the parent batch and adds a new child batch C with the next sequential step's jobs.

Admittedly this process is poorly documented. Check out these code snippets and if you get something that works for you, I'd be happy to add any code example to the wiki to help people in the future.

https://github.com/contribsys/faktory_worker_go/blob/b190f42786c86b871b997314951e604468f1b39a/test/main.go#L174 https://github.com/contribsys/faktory/blob/10ccc2270dc2a1c95c3583f7c291a51b0292bb62/client/batch.go#L189