[DuckDB] Discrepencies between database version read/write

[Ben8t]

Expected Behavior

Last version of DuckDB allows Backward Compatibility, so it should be possible to read databases written from older version

Actual Behaviour

IO Error: Trying to read a database file with version number 64, but we can only read version 51.
The database file was created with an newer version of DuckDB.

The storage of DuckDB is not yet stable; newer versions of DuckDB cannot read old database files and vice versa.
The storage will be stabilized when version 1.0 releases.

For now, we recommend that you load the database file in a supported version of DuckDB, and use the EXPORT DATABASE command followed by IMPORT DATABASE on the current version of DuckDB.

See the storage page for more information: https://duckdb.org/internals/storage

Steps To Reproduce

Full reproducer

id: sqlmesh
namespace: dev
tasks:

  - id: working_dir
    type: io.kestra.core.tasks.flows.WorkingDirectory
    tasks:

      - id: git_clone
        type: io.kestra.plugin.git.Clone
        url: https://github.com/TobikoData/sqlmesh-examples.git
        branch: main

      - id: sqlmesh
        type: io.kestra.plugin.sqlmesh.cli.SQLMeshCLI
        beforeCommands:
          - cd 001_sushi/1_simple
        commands:
          - sqlmesh plan --auto-apply
          - ls -altr db
        outputFiles:
          - '001_sushi/1_simple/db/sushi-example.db'

  - id: query
    type: io.kestra.plugin.jdbc.duckdb.Query
    inputFiles:
      data.db: "{{ outputs.sqlmesh.outputFiles['001_sushi/1_simple/db/sushi-example.db'] }}"
    sql: |
      ATTACH '{{workingDir }}/data.db';
      SELECT * FROM sushisimple.top_waiters;
    store: true

Environment Information

• Kestra Version:
• Plugin version:
• Operating System (OS / Docker / Kubernetes):
• Java Version (If not docker):

Example flow

No response

[loicmathieu]

We're still using DuckDB driver 0.8.1 as 0.9.0 didn't work OOTB, see https://github.com/kestra-io/plugin-jdbc/issues/165

[smunteankestra]

reproducible, but https://github.com/kestra-io/plugin-jdbc/issues/165