vapor / queues

A queue system for Vapor.
MIT License
169 stars 41 forks source link

How to wait for one job to complete before starting the next one? #128

Closed henrypratt closed 11 months ago

henrypratt commented 11 months ago

I am using an AsyncScheduledJob to queue up 4 jobs, but I need them to be run one after another instead of in parallel. I can't find any information online about how to do this.

MahdiBM commented 11 months ago

You can create a parent job that executes those child jobs one by one.

henrypratt commented 11 months ago

That is what I am trying to do, but can't seem to figure out how.

0xTim commented 11 months ago

In the run of one job, when you get to the end, queue the next job up

henrypratt commented 11 months ago

But jobs are dispatched asynchronously and as far as I know there is no way to do that, unless you mean dispatching the next job from within the previous job? Here is a sample of what I am doing:

// Dispatch job 1
try await context
    .queue
    .dispatch(
        AsyncJob1.self,
        .init(""),
        maxRetryCount: 3
    )

// Dispatch job 2
try await context
    .queue
    .dispatch(
        AsyncJob2.self,
        .init(""),
        maxRetryCount: 3
    )

// Dispatch job 3
try await context
    .queue
    .dispatch(
        AsyncJob3.self,
        .init(""),
        maxRetryCount: 3
    )

// Dispatch job 4
try await context
    .queue
    .dispatch(
        AsyncJob4.self,
        .init(""),
        maxRetryCount: 3
    )

// At this point all jobs are dispatched but none have been completed.

Even though I am using await, the next job gets queued up immediately. Shouldn't it wait for the completion of the job? This is all from within an AsyncScheduledJob by the way.

0xTim commented 11 months ago

unless you mean dispatching the next job from within the previous job

That's exactly what I mean. Jobs are not designed to be sequential or one at a time, a worker will process them as it's available to

henrypratt commented 11 months ago

Ok thanks. So I think at this point it just makes more sense to not run the work as jobs.

0xTim commented 11 months ago

Or just run everything in one job - either works

henrypratt commented 11 months ago

Yeah I am going to run it all in a scheduled job. Thanks!