oban-bg / oban

💎 Robust job processing in Elixir, backed by modern PostgreSQL and SQLite3
https://oban.pro
Apache License 2.0
3.37k stars 314 forks source link

Oban Web: Completed Jobs list gets stuck in an empty state #1114

Closed dmarkow closed 5 months ago

dmarkow commented 5 months ago

Environment

Current Behavior

Sometimes, the Completed Jobs list keeps saying "No jobs match the current set of filters." It doesn't matter if I restart my app, restart my browser, etc. The count in the left column does update in realtime, but nothing I do gets the list of completed jobs back.

CleanShot 2024-06-30 at 14 41 06@2x

Expected Behavior

Completed Jobs shows my completed jobs.

sorentwo commented 5 months ago

@dmarkow This is most likely because you have so many available jobs and so few completed. There aren't any completed jobs in the last 100k, as indicated in the message you highlighted. You can increase the limit to a larger number, or remove the limit entirely with a custom resolver: https://getoban.pro/docs/web/2.10.4/Oban.Web.Resolver.html#c:jobs_query_limit/1

That could look like this:

defmodule MyApp.Resolver do
  @behaviour Oban.Web.Resolver

  def jobs_query_limit(_state), do: :infinity
end

Then, in your router:

scope "/" do
  pipe_through :browser

  oban_dashboard "/oban", resolver: MyApp.Resolver
end
dmarkow commented 5 months ago

@sorentwo Ah, ok. I assumed that meant limit the completed queue to the 100k most recently completed, not that the entire Job table is limited to the most recently added 100k and then filtered to just completed. It still seems odd that if I add 500k jobs, I get zero updates in the Completed Jobs page until the first 400k are done. I'll just set it to :infinity for now and see how that works, but this may trip other people up as well. The jobs_query_limit/1 documentation may need a little clarification added. "Ordering is applied after limiting the query" doesn't make it clear that it's actually applying the limit before filtering to the specific job state.

sorentwo commented 4 months ago

The jobs_query_limit/1 documentation may need a little clarification added. "Ordering is applied after limiting the query" doesn't make it clear that it's actually applying the limit before filtering to the specific job state.

Great point, we'll update the docs.