duckdb / community-extensions

https://community-extensions.duckdb.org
111 stars 19 forks source link

Allow specifying different git_ref depending on DuckDB version #70

Closed carlopi closed 2 weeks ago

carlopi commented 1 month ago

Two semantics are supported for the repo.ref field:

  1. string representing the extension ref, currently used by all extensions already, example:

    ref: 09680a975bcf9e93b5a2f46d3eeb68792d5239c6
  2. map duckdb_version -> extension_ref, example:

    ref:
    v1.0.0: 09680a975bcf9e93b5a2f46d3eeb68792d5239c6
    main: 09680a975bcf9e93b5a2f46d3eeb68792d5239c6

When the lookup is performed in scripts/build.py, implementations is: If string, string is used. If dictionary, and DUCKDB_VERSION in dictionary, use dictionary[DUCKDB_VERSION] Otherwise, if dictionary, and main in dictionary, use dictionary[main]

Else: raise an Error.

Why?

In the run-up to v1.1.0, we want to both allow extensions to keep developing / test for v1.0.0 AND allow extensions to test and get ready for the switch to v1.1.0, possibly providing a different git ref if the one currently in use is not compatible with main/v1.1.0.

Expected usage

We will set-up automatic testing versus main, but I am aware that a few extensions that, as is for our own extensions, will not be compatible on the same git_ref with both v1.0.0 and main (that will become v1.1.0 in a few weeks).

Expected usage is that extensions that needs fixing around API changes or equivalent can fix those in the extension repo, and submit a dictionary-version for the repo.ref with the current reference pointed to v1.0.0 and the upcoming version for main.

Versioning

This is not a shortcut to versioning: if multiple hashes are provided they need to be semantically offer the same feature-set. Once a given duckdb version is not actively supported it has to be dropped from the dictionary.

Adding workflow_call

workflow_call section would allow to implement testing on multiple version on top of this, once this PR lands this can be already doable manually via workflow_dispatch, with the other pieces to be added by a subsequent PR.

carlopi commented 1 month ago

Example of extension that might require fixup: prql depends on a now outdated API, and testing shows it not be compatible: https://github.com/carlopi/community-extensions/actions/runs/10282471833/job/28454374050

Error: /Users/runner/work/community-extensions/community-extensions/src/prql_extension.cpp:128:27: error: no viable overloaded operator[] for type 'unique_ptr<RegisteredStateManager>'
  context.registered_state["prql"] = prql_state;
  ~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~
Error: /Users/runner/work/community-extensions/community-extensions/src/prql_extension.cpp:138:46: error: no member named 'find' in 'duckdb::unique_ptr<duckdb::RegisteredStateManager>'
      auto lookup = context.registered_state.find("prql");
                    ~~~~~~~~~~~~~~~~~~~~~~~~ ^
Error: /Users/runner/work/community-extensions/community-extensions/src/prql_extension.cpp:139:46: error: no member named 'end' in 'duckdb::unique_ptr<duckdb::RegisteredStateManager>'
      if (lookup != context.registered_state.end()) {
                    ~~~~~~~~~~~~~~~~~~~~~~~~ ^
3 errors generated.

(a few other duckdb core extensions had already to fix this same API change)

carlopi commented 2 weeks ago

Outdated by #81.