PrefectHQ / prefect-helm

Helm charts for deploying Prefect Services
Apache License 2.0
93 stars 58 forks source link

Support SQLite database #345

Open mitchnielsen opened 3 months ago

mitchnielsen commented 3 months ago

Summary

In https://github.com/PrefectHQ/prefect-helm/issues/336, we found that the database configuration was centralized within PostgreSQL - meaning database configuration couldn't be provided outside of the postgresql key, which matters if someone wants to use MySQL or even SQLite.

Let's consider providing a new top-level key for configuring the database that's considered if postgresql.enabled=false.

Notes

The Prefect docs mention:

Prefect supports the following databases:

  1. SQLite (default in Prefect): Recommended for lightweight, single-server deployments. SQLite requires essentially no setup.
  2. PostgreSQL: Best for connecting to external databases, but requires additional setup (such as Docker). Prefect uses the pg_trgm extension, so it must be installed and enabled.

Elsewhere, it says:

Use SQLite database for testing only SQLite does not support multiprocessing. For high orchestration volume, use PostgreSQL.

So realisticically, most users should be on PostgreSQL. But the documentation shows you how to use SQLite, so I think it's reasonable to support that configuration in the charts. As mentioned in https://github.com/PrefectHQ/prefect-helm/pull/365, users are already using SQLite with the charts with some workarounds.

Acceptance criteria

mitchnielsen commented 1 month ago

I bet this was indirectly addressed by https://github.com/PrefectHQ/prefect-helm/pull/365, but let's try it to be sure and add an SQLite test case to be sure.