Double encoded JSON payload 500s

[davydog187]

• RabbitMQ version 3.8.3
• Erlang version 22.3

2020-09-08 11:09:59.674 [error] <0.18927.110> CRASH REPORT Process <0.18927.110> with 0 neighbours crashed with reason: no function clause matching rabbit_mgmt_util:props_as_map(<<"{\"auto_delete\":false,\"durable\":true}">>) line 875

2020-09-08 11:09:59.974 [error] <0.17898.110> Ranch listener rabbit_web_dispatch_sup_15672, connection process <0.17898.110>, stream 64 had its request process <0.18936.110> exit with reason function_clause and stacktrace [{rabbit_mgmt_util,props_as_map,[<<"{\"auto_delete\":false,\"durable\":true}">>],[{file,"src/rabbit_mgmt_util.erl"},{line,875}]},{rabbit_mgmt_util,with_vhost_and_props,3,[{file,"src/rabbit_mgmt_util.erl"},{line,890}]},{cowboy_rest,call,3,[{file,"src/cowboy_rest.erl"},{line,1576}]},{cowboy_rest,process_content_type,3,[{file,"src/cowboy_rest.erl"},{line,1100}]},{cowboy_rest,upgrade,4,[{file,"src/cowboy_rest.erl"},{line,288}]},{cowboy_stream_h,execute,3,[{file,"src/cowboy_stream_h.erl"},{line,296}]},{cowboy_stream_h,request_process,3,[{file,"src/cowboy_stream_h.erl"},{line,274}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]

It looks like because its still a binary, its failing to match the function head

props_as_map(Val) when is_map(Val)  -> Val;
props_as_map(null)                  -> #{};
props_as_map(undefined)             -> #{};
props_as_map(Val) when is_list(Val) -> maps:from_list(Val).

I'm not super familiar with try/catch in erlang, but it seems like either the try block doesn't exist in 3.8.3, or it is not matching the :error type.

I could be interested in fixing myself, but I might need a little bit of help.

[davydog187]

Possible solutions

1. Handle the binary value in props_as_map/1, but I'm not sure what would be appropriate to return, if anything
2. Catch the function clause error and send the bad request
3. Handle up the stack somewhere?

[michaelklishin]

@davydog187 can we start with how to reproduce this instead of what the solution should be?

[davydog187]

Background

We accidentally released a regression in our application code that double encoded JSON before sending it to the rabbitmq management API. This manifested itself as a 500 in our rabbitmq deployment, rather than a client error, as I would expect.

How to reproduce

Let's start by double-encoding some JSON, then send it to the queue creation API.

iex(2)> %{auto_delete: false, durable: true} |> Jason.encode!() |> Jason.encode!()
"\"{\\\"auto_delete\\\":false,\\\"durable\\\":true}\""

PUT /api/queues/my-vhost/my-queue
"\"{\\\"auto_delete\\\":false,\\\"durable\\\":true}\""

This will decode as binary rather than a map since it is still valid JSON.

[michaelklishin]

@davydog187 the value you have used is successfully parsed by our JSON parser (JSX):

rabbit_json:decode(<<"\"{\\\"auto_delete\\\":false,\\\"durable\\\":true}\"">>).
% => <<"{\"auto_delete\":false,\"durable\":true}">>

Same result with https://jsonlint.com. I'm afraid this double encoded JSON is still valid JSON as far as the parser goes (a string literal).

if we consider this to be only possible when we get incorrectly encoded JSON, this should be handled when the value is decoded (parsed):

with_vhost_and_props(Fun, ReqData, Context) ->
    case vhost(ReqData) of
        not_found ->
            not_found(rabbit_data_coercion:to_binary("vhost_not_found"),
                      ReqData, Context);
        VHost ->
            {ok, Body, ReqData1} = read_complete_body(ReqData),
            case decode(Body) of
                {ok, Props} ->
                    try
                        Fun(VHost, props_as_map(Props), ReqData1)
                    catch {error, Error} ->
                            bad_request(Error, ReqData1, Context)
                    end;
                {error, Reason} ->
                    bad_request(rabbit_mgmt_format:escape_html_tags(Reason),
                                ReqData1, Context)
            end
    end.

[michaelklishin]

The decoding function has a clause that indicates that the input is not considered to be valid JSON:

decode(<<"">>) ->
    {ok, #{}};
decode(Body) ->
    try
        {ok, rabbit_json:decode(Body)}
    catch error:_ -> {error, not_json}
    end.

I am reluctant to base this on a type check but a new clause can be added there.