goccy / bigquery-emulator

BigQuery emulator server implemented in Go
MIT License
790 stars 104 forks source link

Error when trying to use JS UDF #337

Open Domlenart opened 1 month ago

Domlenart commented 1 month ago

What happened?

I'm trying to create a JS UDF from Python like so:

def create_js_udf(client, dataset):
    client.create_routine(
        bigquery.Routine(
            f"{client.project}.{dataset}.{name}",
            language="JavaScript",
            type_="SCALAR_FUNCTION",
            imported_libraries=["gs://some_bucket/turf.js"],
            body="return true",  # This is not the real body, as seen by error below this is not relevant
            return_type=bigquery.StandardSqlDataType(type_kind=bigquery.StandardSqlTypeNames.BOOL),
        ),
        exists_ok=True,
    )

with bigquery.Client(
        project=bigquery_project_id, client_options=client_options, credentials=AnonymousCredentials()
) as client:
    client.create_dataset(f"test.test", exists_ok=True)
    create_js_udf(client, "test")
    yield client

The emulator is configured like this in compose.yaml:

  bigquery-emulator:
    command: [ "--project", "test-project" ]
    image: ghcr.io/goccy/bigquery-emulator:latest
    platform: linux/x86_64
    ports:
      - 9050:9050

When running python code, I see the following log from the emulator:

bigquery-emulator-1  | 2024-07-15T09:52:22.835Z ERROR   server/handler.go:2037  internalError   {"error": "internalError: unsupported language: JavaScript"}

I have inspected the code in the error and looks like JS UDFs are not supported: https://github.com/goccy/bigquery-emulator/blob/58f8350a496fa19d253610b494a05dd57b48e07e/internal/contentdata/repository.go#L487

What did you expect to happen?

I expected that JS UDFs would work as it's stated they work in the README. image

How can we reproduce it (as minimally and precisely as possible)?

Use the snippets I provided above, or contact me for a full repro; I would gladly jump on a call.

Anything else we need to know?

No response

anonimitoraf commented 1 month ago

Did you try using Javascript instead of JavaScript?

Domlenart commented 1 month ago

Yes, the enum with valid languages is here:

https://github.com/goccy/bigquery-emulator/blob/58f8350a496fa19d253610b494a05dd57b48e07e/internal/contentdata/repository.go#L458

Nb, this is also a small bug since, per BQ/ZSQ docs, the valid parameter for JS lang is "JAVASCRIPT".