Apply migrations before waiting for Hasura.

[2start]

Problem: The current docs suggest to extend the user schema by creating a new table in schema public with Hasura and link via a FK to the auth.user schema (See Docs). This does not work because Nhost and Hasura may have a circular dependency on each other if you create a new environment from scratch:

Hasura migrations may depend on auth.user. Auth Service depends on Hasura to be healthy first (i.e. having migrations applied) because the order of execution is as follows:

await waitForHasura(); await applyMigrations(); await applyMetadata();

Solution:

Just change the order of execution to:

await applyMigrations(); await waitForHasura(); await applyMetadata();

This works because the migrations have no dependency on Hasura at all. It fixed our problem with this circular dependency.

Also curious how this was not noticed until now. It's basically not possible to build a new environment from scratch with existing metadata/migrations when referencing auth.user in the Hasura migrations.

[changeset-bot[bot]]

⚠️ No Changeset found

Latest commit: 8c63fad5622724833c1e6ffae5121bab58427fc8

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[2start]

@plmercereau @elitan

Could you please take a look at this. I put in the effort to write up the problem as concisely as possible. Would be happy to be able to merge this.

[Bernix01]

any update on this?

[dbarrosop]

Sorry, we missed this PR somehow. I don't think this is a problem in hasura-auth. From the looks of it, it looks like:

1. You have a project that references the auth.users table, which is fine
2. When you instantiate that project, you are trying to run the migrations for that project before starting hasura-auth for the first time so the auth schema doesn't exist yet.

This looks to me like a problem on the way operations are being performed. Your application depends on hasura-auth but you are trying to instantiate it before hasura-auth.

Also curious how this was not noticed until now

I am guessing you must be self-hosting because this issue doesn't occur with our cli or cloud offering as things are instantiated in the correct order:

1. Start backend services; postgres, hasura, auth, storage
2. Run application's migrations and metadata

Please, let me know if I am misunderstanding the problem.

[xmlking]

I also experienced same problem. I have to run extra sql migrations manually after servers start. looking for advice to automating this https://github.com/xmlking/spectacular/blob/301f260994f133ab8175857c5af9ff6635f70d2e/nhost/migrations/default/1684206403640_init/up.sql#L242

[dbarrosop]

Closing this PR as we don't have plans to merge it, this is a deployment issue.

@xmlking refer to my previous comment. This is an ordering issue in your deployment pipeline. If you application depends on some other service you will need to make sure that other service is up first.

[xmlking]

Thanks I will try adding my migrations container depend on auth and hasura and share the results

[vanHekthor]

@xmlking were you able to resolve your issue? For me, this really does not seem like a deployment issue - at least in the way described here. I feel like this PR was prematurely closed.

Here is my understanding of the issue: The docs clearly suggest referencing the auth.users table. @dbarrosop You are also saying that is fine.

You have a project that references the auth.users table, which is fine

This means there has to be a migration which creates the table referencing auth.users in the public schema.

For this migration to be applied, the auth schema needs to be present. Therefore hasura-auth has to be instantiated.

But hasura-auth itself waits for hasura, the graphql-engine to be ready (see here).

Hasura is never ready, because the aforementioned migration can't be applied.

And there is our deadlock: Hasura can't apply all migrations because the auth schema is missing -> the auth schema is missing because hasura-auth is not instantiated -> hasura-auth is not instantiated because it waits for hasura to be ready -> hasura is not ready because it can't apply all its migrations without the auth schema -> the auth schema ...

This was already addressed in this issue, where Tiim suggests:

It would be nice if hasura-auth could apply the database migrations even if hasura has not started yet.

This PR by @2start proposes exactly this change and it should work because as 2Start says the hasura-auth migrations have no dependency to hasura at all.

[xmlking]

@xmlking were you able to resolve your issue? For me, this really does not seem like a deployment issue - at least in the way described here. I feel like this PR was prematurely closed.

Here is my understanding of the issue: The docs clearly suggest referencing the auth.users table. @dbarrosop You are also saying that is fine.

You have a project that references the auth.users table, which is fine

This means there has to be a migration which creates the table referencing auth.users in the public schema.

For this migration to be applied, the auth schema needs to be present. Therefore hasura-auth has to be instantiated.

But hasura-auth itself waits for hasura, the graphql-engine to be ready (see here).

Hasura is never ready, because the aforementioned migration can't be applied.

And there is our deadlock: Hasura can't apply all migrations because the auth schema is missing -> the auth schema is missing because hasura-auth is not instantiated -> hasura-auth is not instantiated because it waits for hasura to be ready -> hasura is not ready because it can't apply all its migrations without the auth schema -> the auth schema ...

This was already addressed in this issue, where Tiim suggests:

It would be nice if hasura-auth could apply the database migrations even if hasura has not started yet.

This PR by @2start proposes exactly this change and it should work because as 2Start says the hasura-auth migrations have no dependency to hasura at all.

We have to orchestrate the start order of 1. hsura-graphql 2. Hasura-auth 3. Hasura-migration services. Check here https://github.com/xmlking/spectacular/blob/ce748bec72fbef3bd815a6b88e27ea3fcdf6ed2d/compose.override.yml#L51

[2start]

yeah I think this is not a deployment issue but a circular dependency that makes it hard to automate migrations properly.

Let's say you create a migration for a table user_details in schema public and link via a foreign key to auth.users. As soon as you do that you cannot set your project up properly anymore. Hasura will not start because user_details depends on auth.users. Auth service will not create auth.users because Hasura didn't start.

We forked nhost auth and used the solution from this PR. Then, it works because the auth service can start first, apply his migrations to create auth.users and then Hasura can start and create user_details which link to auth.users.