sorentwo / oban

💎 Robust job processing in Elixir, backed by modern PostgreSQL and SQLite3
https://getoban.pro
Apache License 2.0
3.18k stars 297 forks source link

`not_null_violation` Postgres error for Oban scheduled job #1061

Closed whatyouhide closed 3 months ago

whatyouhide commented 3 months ago

Environment

CleanShot 2024-03-26 at 13 09 40@2x

Current Behavior

We got many of these errors in Sentry (redacted actual worker name with MyWorker):

ERROR 23502 (not_null_violation) null value in column "id" of relation "oban_jobs" violates not-null constraint

    table: oban_jobs
    column: id

Failing row contains (null, available, default, MyWorker, {}, {}, 0, 1, null, null, null, null, null, null, null, {}, {"uniq_key": 101711374}, null).

The worker is configured like this:

plugins: [
  {Oban.Plugins.Cron,
   crontab: [
     {"* * * * *", MyWorker},
     # ...
   ]}
]

The stacktrace of the error doesn't involve any of our app, only Oban, which is why I thought to report this:

CleanShot 2024-03-26 at 13 16 16@2x

It could totally still be something wrong we're doing, but I figured I'd give this a try here too 🙃

sorentwo commented 3 months ago

Does the original MyWorker have any unique configuration?

whatyouhide commented 3 months ago

@sorentwo ah good question. This is the config for the worker:

use Oban.Pro.Worker, max_attempts: 1, queue: :default
sorentwo commented 3 months ago

Some more follow up questions as I'm not able to recreate the exception:

  1. Was this a rash of errors all at once, or is it an ongoing occurrence?
  2. Is that worker also inserted as a regular job, not via cron?
whatyouhide commented 3 months ago

@sorentwo

  1. No, ongoing occurrence.
  2. No, only via cron.
whatyouhide commented 3 months ago

Hi @sorentwo any updates on this? We're still seeing this, with around 1.4k occurrences in the past 24h if that helps. Let me know if there is any more context I can provide.

sorentwo commented 3 months ago

@whatyouhide We've been traveling since last week. I can check it out now that we've made it to Lisbon 🙁

whatyouhide commented 3 months ago

@sorentwo sounds perfect. Welcome to Europe if you're not from this side of the world! 🇪🇺

sorentwo commented 3 months ago

@whatyouhide After our discussions on Slack determining that this is limited to a staging environment I'm going to close this out. I'm willing to keep troubleshooting the environmental issue, but seeing as we're not able to recreate it I'd rather not have it looming.