Cannot configure postgres server

[nkuz2]

Describe the bug I have configured a postgres database in AWS. I am trying to load R2R for the first time with "r2r serve --docker --config-path=r2r.toml" I am getting the following error:

"service "hatchet-migration" didn't complete successfully: exit 1
Waiting for all services to become healthy...
r2r does not have a health check defined."

When I look at the compose file for hatchet, I see that DATABASE_URL is: DATABASE_URL: "postgres://${POSTGRES_USER:-postgres}:${POSTGRES_PASSWORD:-postgres}@postgres:5432/${POSTGRES_DBNAME:-postgres}?sslmode=disable"

So, my host may not be substituted in because it is not a variable. How can I resolve this issue?

To Reproduce Steps to reproduce the behavior:

1. Set up a posgres DB on AWS with the correct permissions, ensuring you can ping it from a SageMaker notebook
2. pip install r2r
3. edit r2r.toml to be below.
4. run r2r serve --docker --config-path=r2r.toml"
5. see error

Expected behavior A clear and concise description of what you expected to happen.

Screenshots The log for hatchet migration looks like this, with username and password as expected:

sh-4.2$ docker logs b354c0fb6013
Waiting for database to be ready...
Terminated
DATABASE_URL: postgres://USERNAME:PASSWORD@postgres:5432/postgres?sslmode=disable
Migration name: 
No prisma migration found. Applying migrations via atlas...
Error: postgres: scanning system variables: pq: password authentication failed for user "postgres"
A new version of Atlas is available (v0.27.0): https://github.com/ariga/atlas/releases/tag/v0.27.0
Migration failed. Exiting...
Waiting for database to be ready...
Terminated
DATABASE_URL: postgres://USERNAME:PASSWORD@postgres:5432/postgres?sslmode=disable

My r2r.toml looks like this: [database] user = "USERNAME" password = "PASSWORD" host = "database-1.IDENTIFIER.rds.amazonaws.com" port = "5432" db_name = "postgres"

[chunking] provider = "r2r"

[embedding] provider = "litellm" base_model = "text-embedding-3-large" base_dimension = 1536

[completion] [completion.generation_config] model = "bedrock/mistral.mixtral-8x7b-instruct-v0:"

Desktop (please complete the following information): Accessing from a jupyter notebook in SageMaker, running on Amazon Linux 2.

Smartphone (please complete the following information): N/A

Additional context Add any other context about the problem here.

[NolanTrem]

Hey @nkuz2 when serving with Docker you'll need to set these as environment variables. I'll update the docs to reflect this more clearly and add it to our roadmap to ensure that you can solely set the variables in the config.

[nkuz2]

Hi, I actually set both up in an attempt to get it to work before starting this issue. I am still running into this problem with the following environment variables: export POSTGRES_USER=USERNAME export POSTGRES_PASSWORD=PASSWORD export POSTGRES_HOST=database-1.IDENTIFIER.rds.amazonaws.com export POSTGRES_PORT=5432 export POSTGRES_DBNAME=postgres export DATABASE_URL=postgresql+psycopg2://USERNAME:PASSWORD!1@database-1.IDENTIFIER.amazonaws.com:5432/postgres

[NolanTrem]

I see, I'm able to replicate this with a new Supabase instance. The Hatchet migration script fails, meaning the Hatchet container is never started, which blocks the R2R container.

2024-09-17 20:37:09 Migration failed. Exiting...
2024-09-17 20:37:08 Terminated
2024-09-17 20:37:09 Error: postgres: scanning system variables: pq: password authentication failed for user "postgres.vvumivctmdmeyultndrv"

I'll dig into why this is failing and will hopefully have a fix soon. Thanks!

[nkuz2]

@NolanTrem Seems like they have a fix in the comments: https://github.com/ariga/atlas/issues/2601 Thanks for the quick turnaround!

[NolanTrem]

I've pinged the Hatched team and hopefully they'll add this in shortly!

[igormis]

Any feedback from ✘ Container r2r-hatchet-migration-1 service "hatchet-migration" didn't complete successfully: exit 1

[nkuz2]

Hi @NolanTrem, any updates on this yet? If not, has hatched provided some ETA?

[NolanTrem]

We've not been able to get it resolved so far, unfortunately. We're working on building out our own lightweight orchestration provider to mitigate these problems. Hoping that we'll have this in this week.

[NolanTrem]

@nkuz2 @igormis We've just released a new version that doesn't require Hatchet, making it much easier for you to point to your own Postgres database. I'm going to go ahead and close this out now.

Hope this helps!