sourceName (for nested json attributes) does not seem to work

[wallflower]

Hello,

I was trying to ingest a pretty simple kafka message:

{
  "mqtt_topic": "SOME/MQTT/TEST",
  "received_ts_utc": "2021-07-15T18:36:58.711+00:00",
  "payload": {
    "timestamp": "2021-07-15T18:36:57.123+00:00",
    "value": 9331.516
  }
}

With the following dim configuration:

"dims": [
            {
                "name": "recordtime",
                "type": "String",
                "sourceName": "payload.timestamp"
            },
            {
                "name": "mqtt_topic",
                "type": "String"
            },
            {
                "name": "value_text",
                "type": "String",
                "sourceName": "payload.value"
            },
            {
                "name": "received_ts",
                "type": "Date",
                "sourceName": "received_ts_utc"
            }
        ]

From my understanding of these issues, this should work, but in clickhouse I only see received_ts and mqtt_topic populated, the nested payload attributes come across NULL. so it seems like the nested sourceName is not working as described.

I have looked at: https://github.com/housepower/clickhouse_sinker/pull/56 and https://github.com/housepower/clickhouse_sinker/commit/a6d9097b03efecc4b28818616bba57f19529122c

Any thoughts as to what I am doing wrong?

[yuzhichang]

@wallflower gjson supports nested json attributes, fastjson doesn't. fastjson is the default json parser. Please try gjson. This feature seems broken.

[wallflower]

Sorry for the late reply. Using gjson worked. Thanks!