Swallowed (Postgrex?) exception in Oban.insert_all

Environment

Oban Version - 2.18.3 , oban_pro 1.4.13
PostgreSQL Version - 14.13
Elixir & Erlang/OTP Versions - Elixir 1.17.2, Erlang 27.0.1
ecto - 3.12.4
ecto_sql - 3.12.1
postgrex - 0.19.2

Current Behavior

When using Oban.insert_all with a very big amount of jobs (around 35k), we've seen this not very helpful error:

RuntimeError: operation :rollback is rolling back unexpectedly.

This can happen if `repo.rollback/1` is manually called, which is not supported by `Ecto.Multi`. It can also occur if a nested transaction has rolled back and its error is not bubbled up to the outer multi. Nested transactions are discouraged when using `Ecto.Multi`. Consider flattening out the transaction instead.

  File "lib/ecto/repo/transaction.ex", line 26, in Ecto.Repo.Transaction.transaction/4
  File "lib/oban/repo.ex", line 156, in Oban.Repo.transaction/4
  File "lib/oban/pro/engines/smart.ex", line 733, in anonymous fn/3 in Oban.Pro.Engines.Smart.inner_insert_all/4
  File "lib/enum.ex", line 4353, in Enum.flat_map_list/2
  File "lib/enum.ex", line 4354, in Enum.flat_map_list/2
  File "lib/ecto/adapters/sql.ex", line 1400, in anonymous fn/3 in Ecto.Adapters.SQL.checkout_or_transaction/4
  File "lib/db_connection.ex", line 1756, in DBConnection.run_transaction/4
  File "lib/oban/repo.ex", line 156, in Oban.Repo.transaction/4

The message says that possibly Repo.rollback is mis-used somewhere in oban or oban_pro code? This is a problem because the actual inner issue is not visible anywhere in the exception message or stacktrace, so it's hard to fix. We think for us it was some kind of timeout, possibly a Postgrex exception that was rescued, because we stopped seeing that error after passing timeout: :infinity to insert_all.

Expected Behavior

I think maybe the inner exception should be simply re-raised? Or maybe Multi has some other go-to way of handling such errors without manually calling Repo.rollback

oban-bg / oban