sorentwo / oban

💎 Robust job processing in Elixir, backed by modern PostgreSQL and SQLite3
https://getoban.pro
Apache License 2.0
3.18k stars 297 forks source link

Structs cannot be passed to Job.new #1038

Closed scottmessinger closed 4 months ago

scottmessinger commented 4 months ago

Environment

Current Behavior

In Oban 2.8, the following worked:

      %MyApp.Jobs.Update{cardId: card.id, googleClassId: class_id}
      |> MyApp.Jobs.Update.new(meta: %{cardTitle: card.attributes.title, cardStackId: card_stack_id})
      |> Oban.insert()

Expected Behavior

In Oban 2.17, I now get the following error:

     ** (Protocol.UndefinedError) protocol Enumerable not implemented for %MyAppJobs.Update{....}

Suggested change:

Enable the old behavior so that we can pass structs into the new function. As it is in 2.17, we have to pass raw maps, which means we can't ensure we have the right keys.

sorentwo commented 4 months ago

Job.new/1 has never accepted a struct, only maps:

Application.spec(:oban, :vsn)
# => '2.8.0'

defmodule MyStruct, do: defstruct([:a, :b])
# => {:module, MyStruct, ...

%MyStruct{a: 1, b: 2} |> Oban.Job.new(worker: "SomeWorker") |> Oban.insert()
# => ** (Protocol.UndefinedError) protocol Jason.Encoder not implemented for %MyStruct{a: 1, b: 2} of type MyStruct (a struct), Jason.Encoder protocol must always be explicitly implemented.

The limitation is in Jason.Encoder, and you can derive an implementation for your struct:

defmodule MyStruct do
  @derive Jason.Encoder
  defstruct [:a, :b]
end

Note that deriving a struct will ensure you have the correct keys on insertion, but unlike Pro workers, it won't validate the data types or decode the data back into a struct when the job executes.

scottmessinger commented 4 months ago

I already had @derive Jason.Encoder and it was working in 2.8. It is not working in 2.17.

Note that deriving a struct will ensure you have the correct keys on insertion, but unlike Pro workers, it won't validate the data types or decode the data back into a struct when the job executes.

"won't validate the data types" is this at execution time? I'd like to ensure I have the right data when I create the job, not when it blows up later in the queue. Is there a way to do both?

sorentwo commented 4 months ago

it was working in 2.8. It is not working in 2.17.

Maybe it is related to some other dependency or language changes? It still works with @derive in v2.17:

Application.spec(:oban, :vsn)
~c"2.17.4"

%MyStruct{a: 1, b: 2} |> Oban.Job.new(worker: "SomeWorker") |> Oban.insert()
{:ok, %Oban.Job{...}}

"won't validate the data types" is this at execution time? I'd like to ensure I have the right data when I create the job, not when it blows up later in the queue.

A struct won't validate data types at all, but the args_schema will validate keys and data at insertion time, before it hits the queue.