search

brunostjohn / perplexideez

Search the web and your self-hosted apps using local AI agents.
GNU Affero General Public License v3.0
126 stars 2 forks source link

That's a lot of database containers - might be worth simplifying? #4

Closed sammcj closed 4 hours ago

sammcj commented 4 hours ago

Neat project!

Is your feature request related to a problem? Please describe.

I went to deploy the containers from your compose example and was a little surprised at the number of DBs required.

Describe the solution you'd like

I would have thought you could do it all with one postgres container using postgres's k/v for the k/v store, it's pretty alright at that.

Alternatively with two containers using postgres and redis/valkey if required.

For the db migrations / init you'd usually have your migrations run from the app, when your app starts up if the DB is not up to date.

For the postgres init, if you can't do that with the apps migrations as well, the postgresql container has a directory that if you mount any init scripts - they will be run.

For example /docker-entrypoint-initdb.d/init-user-db.sh:

#!/bin/bash
set -e

psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" <<-EOSQL
    CREATE USER docker;
    CREATE DATABASE docker;
    GRANT ALL PRIVILEGES ON DATABASE docker TO docker;
EOSQL

See https://github.com/docker-library/docs/blob/master/postgres/README.md#initialization-scripts

Describe alternatives you've considered

N/A

Additional context

N/A

brunostjohn commented 4 hours ago

Hi!

Let me explain this so it makes a bit more sense.

The Redis container exists strictly for SearXNG. This app doesn't use it at all and it's not within the scope of the project to modify SearXNG to use Postgres K/V instead. It would be worth a go to refer to the SearXNG team here and ask them for guidance on adding a new K/V adapter. If that does come to fruition, I'm open to PRs with different variants of the current compose stack.

The init_db container exists solely to initialise the Postgres database and chmod pgdata to run Postgres without root privileges (the provided compose manifests already set it up to do so and run Postgres as UID 2000). If you prefer, you can refactor this out in your setup and run Postgres as root, but that has some security implications.

The migrations container is to fetch less data overall. The migration container has a certain size and with Kubernetes setup it will later on be configured to be a Job. This way, not all nodes need to fetch this image in a scenario where the app is running with multiple replicas. This way migrations and the app can also be separately built so users don't need to fetch extra data every time the app updates and the migrations do not (the Prisma CLI required to perform these migrations takes up a bit of space). It is not planned to remove the migrate containers.

Minimising the amount of database containers is not in the scope of the project. The database configuration is entirely up to the user and as long as the database migrations have been applied and there is a database, it is up to the user how it is configured. The provided compose manifests are only an example, but I wouldn't hope to see a stack with a minimised amount of images/an AIO image as most of them are either ephemeral or required. If you prefer something else though, you're welcome to make changes to it. I mentioned the app is flexible in how its ran as I know people have preferences. Theoretically, you can use the Prisma CLI to apply migrations yourself and entirely skip the migration container.

I appreciate the suggestion though! It's a good shout, just doesn't really align with the direction I want to take this project in. With deployments, I assume the users can deploy this themselves, so I provide a robust-ish setup as a starting point upon which I expect the user to slightly expand. I hope that makes it make more sense.