mgcrea / prisma-queue

Minimalist postgresql job queue for Prisma
MIT License
53 stars 5 forks source link

Different timezone for `processedAt` #7

Closed noxify closed 3 months ago

noxify commented 4 months ago

Hey,

checked the queues_job table and have seen that there is a mismatch in the timezones.

Here an example ( exported from the db )

  {
    "id": 9,
    "queue": "notification",
    "key": null,
    "cron": null,
    "payload": "{...}",
    "result": "{\"status\": 200}",
    "error": null,
    "progress": 100,
    "priority": 0,
    "attempts": 1,
    "maxAttempts": null,
    "runAt": "2024-07-08 07:59:59.195",
    "notBefore": null,
    "finishedAt": "2024-07-08 08:00:03.385",
    "processedAt": "2024-07-08 10:00:03.371",
    "failedAt": null,
    "createdAt": "2024-07-08 07:59:59.195",
    "updatedAt": "2024-07-08 08:00:03.369"
  }

I'm located in Timezone "Europe/Berlin" (GMT+0200) and have triggered the queue at 2024-07-08 10:00:03.371 local time. Most of the date/time values are in UTC format which is fine, but the processedAt seems wrong for me ( local time instead of utc ).

Based on the code, the field is filled here: https://github.com/mgcrea/prisma-queue/blob/master/src/PrismaQueue.ts#L307

mgcrea commented 4 months ago

What is the timezone of your database? (SHOW TIME ZONE) You can try to pass the alignTimeZone option to true when creating the queue to automatically align it from the Node.JS host process.

Not sure NOW() AT TIME ZONE 'utc' is the proper fix here but will investigate a bit.

noxify commented 4 months ago

Hi,

thanks for the hints.

What is the timezone of your database?

SHOW TIME ZONE
# Europe/Berlin

You can try to pass the alignTimeZone option to true when creating the queue to automatically align it from the Node.JS host process.

Tried it, but the result was the same.

Checked also the timezone in my shell

❯ date +%Z
CEST

Which seems to be the same as in the database.

Have seen that you check the timezone in the dequeue function. I have activated the debug prints via DEBUG=prisma-queue, but I got no debug print for the assignment. So I assume that there is no mismatch between the two timezones ( DB + app level ).


Here the snippet for the queue creation:

// packages/queue/src/notification.ts
import { createQueue } from "@mgcrea/prisma-queue"

import { db } from "@acme/db"
import { createLogger } from "@acme/logging"

// eslint-disable-next-line @typescript-eslint/consistent-type-definitions
type JobPayload = {
  type: "create" | "update"
  relatedTabe: "application" | "environment"
  relatedId: string
}

// eslint-disable-next-line @typescript-eslint/consistent-type-definitions
type JobResult = {
  status: number
}

const logger = createLogger()

const queueName = "notification"

export default createQueue<JobPayload, JobResult>(
  {
    name: queueName,
    maxConcurrency: 5,
    prisma: db,
    maxAttempts: 3,
    alignTimeZone: true,
  },
  async (job) => {
    const { id, payload } = job
    logger.withContext({ queueName, id, payload }).debug(`Processing job`)
    await job.progress(50)
    const status = 200
    if (Math.random() > 0.5) {
      throw new Error(`Failed for some unknown reason`)
    }
    logger.withContext({ queueName, id, payload }).debug(`Finished job`)
    return { status }
  },
)

Inside the job manager app, I run the following script to start the queues ( I have multiple queue and the plan is to use PM2 to start and monitor them )

import yargs from "yargs"
import { hideBin } from "yargs/helpers"

import type { AvailableQueue } from "@acme/queue"
import { createLogger } from "@acme/logging"
import queues from "@acme/queue"

const logger = createLogger()

const args = yargs(hideBin(process.argv))
  .choices("queue", Object.keys(queues))
  // .demandOption("queue")
  // .fail(() => {
  //   process.exit(1)
  // })
  .parseSync()

const startQueue = async (queue: AvailableQueue = "notification") => {
  logger.withContext({ queue }).info("start queue")
  await queues[queue].start()
}

void startQueue(args.queue as AvailableQueue)

Note: import queues from "@acme/queue" is just an exported object with all available queues.

// packages/queue/src/index.ts
import notification from './notification'
export default {
  notification,
}

If you need more information or something else, please let me know.

noxify commented 3 months ago

Hi @mgcrea,

TL;DR: It seems to be a local problem - It works fine in the deployed aws rds instance.


we made some progress with our project and having a running ci/cd pipeline which deploys a postgres 16 rds ( we're using AWS as host ).

I tested it again in the deployed version and it seems that everything is working as expected. To see the data in the remote rds, I have used DBEaver ( locally, I use TablePlus and DBngin ).

Will close this issue, since it works where it has to work ;)

Thanks again for taking the time while analyzing the issue ✌️