Datasources and persistence units with multiple "alternatives" listed at build time and selected at runtime

yrodiere commented 1 day ago

Description

Problem statement

Some applications are built once, then distributed as a single binary (container image) and consumed/deployed by third-parties (users, customers) in various environments.

Such binaries need to work in multiple environments, and that means in particular that datasources and persistence units may need to work with different database kinds (e.g. Oracle and Postgres) depending on the deployment.

Currently, application developers have two solutions to address this:

Declare multiple named persistence units and datasources, one per db-kind, make them active/inactive based on configuration profiles, have a custom PU/datasource CDI bean whose producer would have point to the active PU/datasource at runtime (see previous link).
OR produce a "mutable jar" and have users/customers execute re-augmentation.

This has several downsides, however:

Either solution can be quite complex to implement for application developers.
Solution 1 implies a distributed container image that contains all the alternatives -- but that's the bare minimum to address such use case.
Solution 1 doesn't work with Panache -- which requires each entity to be bound to at most one persistence unit.
Solution 2 implies a distributed container image that is much larger (includes deployment JARs and who knows what else needed for reaugmentation).
Solution 2 implies a intermediary "rebuild" step that will either slow down startup and make the container image much larger (if executed on container startup) or make consumption of the container image much more complex (if executed manually by the users/customers to derive a final container image).
Solution 2 is not compatible with native builds.

Proposal

We could expose more guided, better integrated ways of developing and deploying of such applications.

Implementation ideas

If we prefer solution 1 above (active/inactive)

We could provide a built-in concept of "alternatives" in the configuration of persistence units or datasources:

# Defining the default datasource with multiple alternatives.
# In practice the extension will generate one set of bytecode/bean per alternative,
# and use only one at runtime.
quarkus.datasource.jdbc.min-size=10
quarkus.datasource.alternatives."db2".db-kind=db2
quarkus.datasource.alternatives."db2".jdbc.max-size=60
quarkus.datasource.alternatives."postgresql".db-kind=postgresql
quarkus.datasource.alternatives."postgresql".jdbc.max-size=40

# Runtime profiles allow selecting an alternative
%db2.quarkus.datasource.selected-alternative=db2
%pg.quarkus.datasource.selected-alternative=postgresql

# Hibernate ORM consumes the default datasource,
# so it will automatically generate one alternative per datasource,
# and automatically select the correct one depending on the selected datasource at runtime
quarkus.hibernate-orm.database.generation=create

Main downsides:

This still implies a distributed container image that contains all the alternatives -- but that's still the bare minimum to address such use case.
This is very ad-hoc and requires per-extension implementation. We may provide some basic scaffoding in Quarkus (e.g. standard build items and helpers around configuration), but most of the heavy lifting will have to be implemented per extension.
\

If we prefer solution 2 above (re-augmentation)

We could provide a simple process and technical solution that could be enabled simply (e.g. configuration flag) by application developers, and could by deployed simply by users/customers, triggering re-augmentation only once:

We could have Quarkus generate a multi-stage docker file for users/consumers to consume. Perhaps it is simple enough for Openshift (which I think allows defining the application to deploy using a Dockerfile), but it probably isn't for everyday docker/podman/testcontainers usage.
We could have Quarkus generate an operator as part of the build, whose only task would be to trigger the build and. Not sure if it makes sense or if it's even feasible, though.
Open to other ideas... ?

Main downsides:

This is still quite complex.
This still implies a slow first deployment.
This still requires large distributed applications (they include deployment JARs).
This still wouldn't support native builds.

cc @maxandersen @cescoffier @geoand @gsmet and anyone you think would be interested: your ideas could help here...

quarkus-bot[bot] commented 1 day ago

/cc @geoand (openshift), @iocanel (openshift)

FroMage commented 1 day ago

Solution 1 doesn't work with Panache

See for a possible solution for Panache 1 and repositories https://github.com/quarkusio/quarkus/issues/43341

Panache 2 should have something better, to also make this possible.

quarkusio / quarkus