edgurgel / verk

A job processing system that just verks! 🧛‍
https://hex.pm/packages/verk
MIT License
723 stars 65 forks source link

Scheduled zero arg jobs fail #149

Closed nsweeting closed 6 years ago

nsweeting commented 6 years ago

It appears as though scheduled zero-arg jobs end up getting their [] arg list cast into a {} with the LPUSH command.

Verk.Supervisor.start_link

defmodule Job do
  def perform do
    IO.inspect "hello"
  end
end

job = %Verk.Job{class: Job, args: [], queue: "default"}

Currently this results in:

15:02:39.544 [info]  Elixir.Job 13837351251859802746 start
15:02:39.548 [debug] Job failed reason: %ArgumentError{message: "argument error"}
15:02:39.548 [info]  Elixir.Job 13837351251859802746 fail: 5 ms
15:02:39.552 [error] GenServer #PID<0.196.0> terminating
** (stop) :failed
Last message: {:"$gen_cast", {:perform, %Verk.Job{args: %{}, class: "Elixir.Job", enqueued_at: nil, error_backtrace: nil, error_message: nil, failed_at: nil, finished_at: nil, jid: "13837351251859802746", max_retry_count: 25, original_json: "{\"error_message\":null,\"max_retry_count\":25,\"jid\":\"13837351251859802746\",\"class\":\"Elixir.Job\",\"retried_at\":null,\"queue\":\"default\",\"finished_at\":null,\"enqueued_at\":null,\"error_backtrace\":null,\"args\":{},\"failed_at\":null,\"retry_count\":0}", queue: "default", retried_at: nil, retry_count: 0}, #PID<0.197.0>}}
State: nil

And the redis console output:

1518724956.160306 [0 127.0.0.1:54702] "ZADD" "schedule" "1518724956" "{\"retry_count\":0,\"retried_at\":null,\"queue\":\"default\",\"max_retry_count\":25,\"jid\":\"13837351251859802746\",\"finished_at\":null,\"failed_at\":null,\"error_message\":null,\"error_backtrace\":null,\"enqueued_at\":null,\"class\":\"Elixir.Job\",\"args\":[]}"
1518724957.987005 [0 lua] "LPUSH" "queue:default" "{\"error_message\":null,\"max_retry_count\":25,\"jid\":\"13837351251859802746\",\"class\":\"Elixir.Job\",\"retried_at\":null,\"queue\":\"default\",\"finished_at\":null,\"enqueued_at\":null,\"error_backtrace\":null,\"args\":{},\"failed_at\":null,\"retry_count\":0}"
edgurgel commented 6 years ago

Thanks for the bug report! Yeah this looks wrong 🤔

krasio commented 6 years ago

Looks like this comes from the cjson package used in the lua scripts:

127.0.0.1:6379> EVAL 'return cjson.encode(cjson.decode(\'{"args": []}\'))' 0
"{\"args\":{}}"

See also https://github.com/mpx/lua-cjson/issues/11#issuecomment-287617268.

Not sure what can be done. 😞

@nsweeting The only workaround I can think of at the moment is to have at least one argument for such jobs even if it's not used.

defmodule Job do
  def perform(1) do
    IO.inspect "hello"
  end
end

job = %Verk.Job{class: Job, args: [1], queue: "default"}

Not Ideal but better than getting that error.

edgurgel commented 6 years ago

Maybe we could try to "patch" the JSON when decoding? We know that args is an array. I wonder what's the best place to do this...

krasio commented 6 years ago

What about something like this - https://github.com/edgurgel/verk/pull/154?