problem with serializing strong params (3.11.0+)

[ellmo]

Problem

Hello there

Seems like I have an issue with oj (after updating to 3.16.4) not properly being used in Rails when it comes to ActionController::Parameters.

Since the users params is passed further to a Sidekiq worker, Sidekiq uses JSON.generate(object) to serialize parameters.

JSON.generate is one of three different ways of doing this incorrectly:

users = params["users"]
#=> [#<ActionController::Parameters {"email"=>"test@test.com", "first_name"=>"joe", "last_name"=>"test"} permitted: false>]

JSON.generate users
#=> "[\"{\\\"email\\\"=>\\\"test@test.com\\\", \\\"first_name\\\"=>\\\"joe\\\", \\\"last_name\\\"=>\\\"test\\\"}\"]"

JSON.dump users
#=> "[\"{\\\"email\\\"=>\\\"test@test.com\\\", \\\"first_name\\\"=>\\\"joe\\\", \\\"last_name\\\"=>\\\"test\\\"}\"]"

Oj.generate users
#=> "[\"{\\\"email\\\"=>\\\"test@test.com\\\", \\\"first_name\\\"=>\\\"joe\\\", \\\"last_name\\\"=>\\\"test\\\"}\"]"

None of those approaches returns a properly serialized object. If you try to parse it back as JSON, it will return:

reverse = Oj.load(Oj.generate users)
#=> ["{\"email\"=>\"test@test.com\", \"first_name\"=>\"joe\", \"last_name\"=>\"test\"}"]

So now, instead of an array of hashes, we end up with an array of strings. And those strings - mind you - cannot be further parsed:

reverse.map {|x| Oj.load(x)}
#=> Oj::ParseError: unexpected character (after email) at line 1, column 9 [parse.c:762]

The only module/method combination that works is:

Oj.dump users
#=> "[{\"email\":\"test@test.com\",\"first_name\":\"joe\",\"last_name\":\"test\"}]"
Oj.load _
#=> [{"email"=>"test@test.com", "first_name"=>"joe", "last_name"=>"test"}]

Question

So here's my question: Is there an option for Oj (3.16.4) and Rails 6.1+ that I'm missing? This was working as expected up to oj 3.10.18 and it breaks with anything over that.

My current config for Oj is this:

MultiJson.use(:oj)

Oj.default_options = {
  bigdecimal_as_decimal: true,
  bigdecimal_load: :auto,
  mode: :custom,
  second_precision: ActiveSupport::JSON::Encoding.time_precision,
  time_format: :xmlschema,
  use_as_json: true,
}
Oj.optimize_rails

[ohler55]

It looks like ActionController::Parameters is not encoding as a json object. I'm not really set up to test this right now so would you be able to help?

The first thing to try would be to see if there is an #as_json or #to_json method on ActionController::Parameters. If so that does it return?

[ellmo]

prying into the controller (using 3.11.8)

params
#=> #<ActionController::Parameters {"url"=>"(...)", "users"=>[#<ActionController::Parameters {"email"=>"test@test.com", "first_name"=>"joe", "last_name"=>"test"} permitted: false>], "uuid"=>"something-something", "format"=>"json", "controller"=>"api/v1/users", "action"=>"create"} permitted: false>

users = params[:users]
#=> [#<ActionController::Parameters {"email"=>"test@test.com", "first_name"=>"joe", "last_name"=>"test"} permitted: false>]

users.as_json
#=> [{"email"=>"test@test.com", "first_name"=>"joe", "last_name"=>"test"}]
users.to_json
#=> "[{\"email\":\"test@test.com\",\"first_name\":\"joe\",\"last_name\":\"test\"}]"

and yet, after sending users to a sidekiq worker:

users
#=> ["{\"email\"=>\"test@test.com\", \"first_name\"=>\"joe\", \"last_name\"=>\"test\"}"]

Is this what you needed?

[ohler55]

What I'm trying to determine is if the ActionController::Parameters.as_json exists and is being called. It kind of looks like #to_json is being called by Oj instead of #as_json.

[ellmo]

Hey, circling back to this – any way I can help?

[ohler55]

What does the json gem emit when calling JSON.generate without Oj?

It appears as if the #to_json method is being called on the params instead of #as_json. It's as if the :use_as_json option is not set which would be the normal case for the JSON gem.