Sending request with non-string value to kafka-enabled endpoint terminates RIG

[actraiser]

Using RIG 2.2.1, when a client sends a request to a RIG Proxy API-Endpoint that should create a Kafka event AND when that request contains a non-string value by either accident or deliberately sent by some bad user, RIG crashes.

Possible solution: when receiving requests on a Kafka-enabled target endpoint, RIG should check that all values are strings and if there is a non-string value, either reject the request or silently toString() all values before forwarding to Kafka.

Example Proxy-Config to accept request to create a Kafka record:

[{
  "id": "kafka",
  "version_data": {
    "default": {
      "endpoints": [{
        "id": "produce",
        "method": "POST",
        "path": "/",
        "target": "kafka"
      }]
    }
  }
}]

Event including a field containing a number (statusCode)

{
  "event": {
    "statusCode": 200,
    "id": "069711bf-3946-4661-984f-c667657b8d85",
    "type": "com.example",
    "time": "2018-04-05T17:31:00Z",
    "specversion": "0.2",
    "source": "/cli",
    "contenttype": "application/json",
    "data": "{\"item\":2,\"other\":\"twelve\"}"
  },
  "partition": "0"
}

Sending the event to the configured Proxy Endpoint will immediately crash RIG. If putting the number in quotes, it works fine.

08:08:58.111 [info]  POST /
request_id=FdZb4CkQCpiJl0QAAADS application=plug module=Plug.Logger function=call/2 file=lib/plug/logger.ex line=27 pid=<0.3312.0>
:gen_event handler Logger.Backends.Console installed in Logger terminating
** (exit) an exception was raised:
    ** (Protocol.UndefinedError) protocol String.Chars not implemented for {%ArgumentError{message: "argument error"}, [{:erlang, :iolist_to_binary, [200], []}, {:brod_utils, :bin, 1, [file: 'src/brod_utils.erl', line: 648]}, {:brod_utils, :"-unify_msg/1-fun-0-", 1, [file: 'src/brod_utils.erl', line: 657]}, {:lists, :map, 2, [file: 'lists.erl', line: 1239]}, {:lists, :map, 2, [file: 'lists.erl', line: 1239]}, {:brod_utils, :unify_msg, 1, [file: 'src/brod_utils.erl', line: 657]}, {:brod_utils, :make_batch_input, 2, [file: 'src/brod_utils.erl', line: 435]}, {:brod_producer, :produce_cb, 4, [file: 'src/brod_producer.erl', line: 199]}]}. This protocol is implemented for: Atom, BitString, Date, DateTime, Float, Integer, List, NaiveDateTime, Rig.Subscription, Time, URI, Version, Version.Requirement
        (elixir) /usr/local/src/elixir/lib/elixir/lib/string/chars.ex:3: String.Chars.impl_for!/1
        (elixir) /usr/local/src/elixir/lib/elixir/lib/string/chars.ex:22: String.Chars.to_string/1
        (logger) lib/logger/formatter.ex:166: anonymous fn/1 in Logger.Formatter.output/5
        (elixir) lib/enum.ex:1314: Enum."-map/2-lists^map/1-0-"/2
        (elixir) lib/enum.ex:1314: Enum."-map/2-lists^map/1-0-"/2
        (logger) lib/logger/formatter.ex:152: anonymous fn/6 in Logger.Formatter.format/5
        (elixir) lib/enum.ex:1925: Enum."-reduce/3-lists^foldl/2-0-"/3
        (logger) lib/logger/formatter.ex:151: Logger.Formatter.format/5

08:08:58.154 [info]  Application rig_kafka exited: shutdown
module=application_controller function=info_exited/3 file=application_controller.erl line=1941 pid=<0.1744.0>
{"Kernel pid terminated",application_controller,"{application_terminated,rig_kafka,shutdown}"}
Kernel pid terminated (application_controller) ({application_terminated,rig_kafka,shutdown})

Crash dump is being written to: erl_crash.dump...done

[mmacai]

@actraiser thanks for the report, I'll check!

[mmacai]

@actraiser tried both 2.1.1 and latest and was able to send the mentioned event successfully. Would it be possible for you to create some small example where it is reproduced?

What I've used:

[
  {
    "id": "my-service",
    "version_data": {
      "default": {
        "endpoints": [
          {
            "id": "my-endpoint",
            "method": "POST",
            "path": "/",
            "target": "kafka"
          }
        ]
      }
    },
    "proxy": {
      "use_env": true,
      "target_url": "API_HOST",
      "port": 3000
    }
  }
]

docker run --name rig -d \
-e PROXY_CONFIG_FILE="$config" \
-e KAFKA_BROKERS="kafka:9092" \
-e PROXY_KAFKA_REQUEST_TOPIC="req-topic" \
-e LOG_LEVEL=debug \
-p 4000:4000 \
--network kafka_default \
accenture/reactive-interaction-gateway:2.2.1

[actraiser]

Thanks for checking! I will review my code after the weekend again and let you know how I triggered those crashes.

I actually believe that I made a wrong assumption of the underlying problem in the first place and that those crashes rather happen when using the Kafka-Endpoint in conjunction with an AVRO-Schema.

When encoding fails (because the sent data does not match the schema fetched from the schema-registry), rig seems to crash - but I will go recheck and and provide a reproducable case.

Greets -act

[actraiser]

Sorry for the delay - I had to reinstall my environment (thank you Apple) but ever since, I can not reproduce the problem any more. Please consider the issue closed.

[mmacai]

Ok, no problem. Feel free to re-open or create new issue if you tackle it again.