Prevent blocking people with toolchain issues

[samansmink]

Currently to publish a community extension, developers are blocked by being forced to use the toolchain from https://github.com/duckdb/extension-ci-tools. We have added some optional build dependencies, and you can use vcpkg, but inevitably this is still blocking some extensions that require other build time dependencies.

To fully resolve this issue we should have a path where people have full control over the toolchain (potentially at the risk of breaking extensions, but this is at their own risk of course).

Currently we support

• [x] keep the standardized toolchain in https://github.com/duckdb/extension-ci-tools, and keep accepting PRs for adding optional components BUT only for components that are generally useful and not too exotic.

To resolve this issue we need to support:

• [x] Allow running a custom script that extends the default toolchain

Potentially in the future we can look into adding:

• [ ] allow a fully custom toolchain, potentially by calling into a user-defined workflow file (does GH actions even allow this?) Alternatively, we could allow creating a tagged release in the extension repository with the binary files and upload those to the community extension repo instead of building it in this repo's CI

[carlopi]

Two things that came to my mind while reading this:

• aren't fully customisable toolchains already available (sort of) by allowing arbitrary CMake files? Those can already install packages / build time dependencies, arguably with a not super portable syntax. Unsure where this leads, but I'd say we already allow quite some flexibility on what's available at build time. Would that solve, at least partially, the problem?
• uploading binary files pre-built by a given repository, without us building the files, has one problem that we gave up some guarantees on reproducibility, so ideally I would like to help devs bring their extension into the main workflows

[JAicewizard]

aren't fully customisable toolchains already available (sort of) by allowing arbitrary CMake files? Those can already install packages / build time dependencies, arguably with a not super portable syntax. Unsure where this leads, but I'd say we already allow quite some flexibility on what's available at build time. Would that solve, at least partially, the problem?

Does this work with vcpkg? I have some vcpkg ports that require system dependencies, (omp and fortran specifically) and I think vcpkg runs before the extension cmake script.

[JAicewizard]

To add, personally I think that having a generic custom extension step, in which all kinds of things can be done (install dependencies, patch some things up, whatever) somewhere in the middle might be a way to allow extensions to have practically all the freedom they need without adding a gazillion options to the workflow. This could just be a .sh/.bat passed in to the workflow as an option. I think it would cover most cases, and I think that the user would have to be malicious in order to do break reproducibility and/or incompatibility.

[samansmink]

@JAicewizard I've implemented your suggestion, see the extension template (quack extension) in this repo for an example