kestra-io / kestra

:zap: Workflow Automation Platform. Orchestrate & Schedule code in any language, run anywhere, 500+ plugins. Alternative to Zapier, Rundeck, Camunda, Airflow...
https://kestra.io
Apache License 2.0
13.68k stars 1.18k forks source link

DuckDB plugin does not support DuckDBArray data type #4303

Open aku opened 4 months ago

aku commented 4 months ago

Describe the issue


java.lang.IllegalArgumentException: Data of type 'class org.duckdb.DuckDBArray' for column '[REDACTED]' is not supported
    at io.kestra.plugin.jdbc.AbstractCellConverter.convert(AbstractCellConverter.java:81)
    at io.kestra.plugin.jdbc.duckdb.DuckDbCellConverter.convertCell(DuckDbCellConverter.java:49)
    at io.kestra.plugin.jdbc.AbstractJdbcQuery.convertCell(AbstractJdbcQuery.java:189)
    at io.kestra.plugin.jdbc.AbstractJdbcQuery.mapResultSetToMap(AbstractJdbcQuery.java:182)
    at io.kestra.plugin.jdbc.AbstractJdbcQuery.fetch(AbstractJdbcQuery.java:167)
    at io.kestra.plugin.jdbc.AbstractJdbcQuery.fetchToFile(AbstractJdbcQuery.java:148)
    at io.kestra.plugin.jdbc.AbstractJdbcQuery.run(AbstractJdbcQuery.java:109)
    at io.kestra.plugin.jdbc.duckdb.Query.run(Query.java:231)
    at io.kestra.plugin.jdbc.duckdb.Query.run(Query.java:32)
    at io.kestra.core.runners.WorkerTaskThread.doRun(WorkerTaskThread.java:77)
    at io.kestra.core.runners.AbstractWorkerThread.run(AbstractWorkerThread.java:56)```

### Environment

- Kestra Version: 17.8
- Operating System (OS/Docker/Kubernetes):
- Java Version (if you don't run kestra in Docker):
anna-geller commented 4 months ago

Could you share a reproducer (example flow that generates that error)? It's difficult to help otherwise. thx in advance!

aku commented 4 months ago

@anna-geller

  - id: duckdb
    type: io.kestra.plugin.jdbc.duckdb.Query
    sql: SELECT [1, 2, 3] AS Result
    fetchOne: true
KunalKumar-1 commented 1 month ago

hey @anna-geller, has this issue been resolved ? if not i could work on it .

anna-geller commented 1 month ago

sure, go ahead and thanks in advance! 👍

KunalKumar-1 commented 1 month ago

@anna-geller , I'm having little difficulty identifying the exact location of the relevant files needed to make the change. Could you please point out to where the io.kestra.plugin.jdbc.duckdb.Query and AbstractCellConverter.convert classes are located in the codebase?

Ben8t commented 1 month ago

Note: unfortunetly we have an upstream issue with DuckDB Java driver opened in DuckDB repo. It seems that array type has been introduced in 0.10 and Kestra plugin version is stuck to 0.8 due to the later issue.