warpstreamlabs / bento

Fancy stream processing made operationally mundane. This repository is a fork of the original project before the license was changed.
https://warpstreamlabs.github.io/bento/
Other
1.07k stars 71 forks source link

Create JSONata processor #165

Open gregfurman opened 1 week ago

gregfurman commented 1 week ago

Context

JSONata is a powerful query language that can be used to operate on JSON data. Namely, it has a very rich feature library that would lend itself well to be used on Bento message data.

Motivation

Task

Create a JSONata processor component (similar to the jq processor) that can operate on Bento message data. The blues/jsonata-go library seems like the best option here.

In addition, this component should:

Note: JSONata does not mutate JSON data, instead creating new JSON objects.

Example usage

Flatten a deeply nested array:

jsonata: |
  $.**
# In: [[[[0]]] ,1 , [2], [3, [4]], 5]
# Out:  [0, 1, 2, 3, 4, 5]

FInd the total cost of orders:

jsonata: |
   $sum(orders.(price*quantity))
# In: 
# {
#   "orders": [
#     {"price": 10, "quantity": 3},
#     {"price": 0.5, "quantity": 10},
#     {"price": 100, "quantity": 1}
#   ]
# }
# Out: 135

Reverse and reduce a list of JSON objects:

jsonata: |
   $reverse($[].id)
# In: [{"id": "id-2"}, {"id": "id-3"}, {"id": "id-4"}, {"id": "id-1"}]
# Out: ["id-1", "id-4", "id-3", "id-2"]

Sum the items array and return a new reduced object:

jsonata: |
   {"total": $sum(items)}
# In: {"items": [7, 8, 6, 5], "name": "Mr Bento"}
# Out: {"total": 26}

Get all cities in WA:

jsonata: |
   {"Cities": locations[state="WA"].name ~>  $sort() ~> $join(", ")}

# In:
# {
#   "locations": [
#     {"name": "Seattle", "state": "WA"},
#     {"name": "New York", "state": "NY"},
#     {"name": "Bellevue", "state": "WA"},
#     {"name": "Olympia", "state": "WA"}
#   ]
# }
# Out: {"Cities": "Bellevue, Olympia, Seattle"}