redpanda-data / connect

Fancy stream processing made operationally mundane
https://docs.redpanda.com/redpanda-connect/about/
8.14k stars 840 forks source link

[`javascript` processor] Update `goja` packages to the latest versions #2910

Closed heikkilamarko closed 1 month ago

heikkilamarko commented 1 month ago

Hi, I tried using the Math.js (https://mathjs.org/) library in the javascript processor:

input:
  generate:
    mapping: |
      root.a = [[1, 2], [3, 4]] 
      root.b = [[5, 6], [1, 1]]
    count: 10
    interval: ""

pipeline:
  processors:
    - javascript:
        code: |
          function run() {
            const math = require("mathjs")
            const root = benthos.v0_msg_as_structured()
            root.result = math.multiply(root.a, root.b)
          }
          run();
output:
  file:
    path: out.json

logger:
  level: DEBUG

However, this gave me an error because the goja version that Redpanda Connect is using is outdated and does not support BigInt. I created a custom build where I updated the goja and goja_nodejs packages to the latest version, and the pipeline started to work.

Screenshot 2024-09-30 at 19 37 57

Is there any chance these packages could be updated, so that a custom build is no longer needed?

heikkilamarko commented 1 month ago

BigInt support was added to goja in this PR (Aug 22): https://github.com/dop251/goja/pull/597