kestra-io / plugin-jdbc

https://kestra.io/plugins/
Apache License 2.0
10 stars 7 forks source link

Prevent multiple SQL statements in the Query tasks #377

Open anna-geller opened 1 week ago

anna-geller commented 1 week ago

Feature description

We currently have some inconsistencies, resulting in some plugins supporting multiple SQL statements (more of a side effect rather than intended behavior) while others support only a single SQL statement.

To simplify + add more consistency, we'll follow the approach:

We should prevent running multiple SQL queries in Query tasks:

Ben8t commented 1 week ago

Have to try again, but running many SQL statements separated by ; should work 🤔 ?

Currently these queries run successfully with postgres.Query for example:

      CREATE SCHEMA IF NOT EXISTS weather;
      CREATE TABLE IF NOT EXISTS weather.tmp (
        "time" VARCHAR,
        "temperature_2m" VARCHAR,
        "latitude" VARCHAR,
        "longitude" VARCHAR,
        "city" VARCHAR
      );

      CREATE TABLE IF NOT EXISTS weather.temperature (
        "time" VARCHAR,
        "temperature_2m" VARCHAR,
        "latitude" VARCHAR,
        "longitude" VARCHAR,
        "city" VARCHAR,
        primary key(time, city)
      );
Ben8t commented 1 week ago

I can definitely see a need to run many queries in one task. Especially DDL queries which are often stored in one .sql file

- id: ddl
  type: postgres.Query
  sql : {{ read('ddl.sql') }} # where ddl.sql contain many SQL statements