hasura / graphql-engine

Blazing fast, instant realtime GraphQL APIs on your DB with fine grained access control, also trigger webhooks on database events.
https://hasura.io
Apache License 2.0
30.95k stars 2.74k forks source link

No change database capture (CDC) on Hasura Cloud #6988

Open mikejcooper opened 3 years ago

mikejcooper commented 3 years ago

Cannot see the database change capture (CDC) in hdb_catalog when applying migration on Hasura Cloud, but works fine on local Hasura deployment.

On Hasura Cloud hdb_catalog has the following tables after 1 migration = [event_log, event_invocations_log, hdb_source_catalog_version]

On local deployment, hdb_catalog has many tables after 1 migration, including hdb_version which contains migration info.

Any ideas?

coco98 commented 3 years ago

@mikejcooper On Hasura Cloud, the metadata storage is now done in Hasura Cloud itself. So the CDC bits (required for event triggers) are the only footprint of Hasura you'll see in your source database.

With the next few releases, that will also become optional so that if you don't have any event triggers, the event_* tables will also not be created. This will allow users to bring in purely read-only databases as sources as well.

On the Hasura-2.0 docker images you'll see the metadata tables being created in the metadata database. Ofcourse, its optional to bring your own separate metadata database when running Hasura with Docker. So if you have one postgres database being used both as a source and for metadata storage, you will see all the hdb_* tables in the hdb_catalog postgres schema.

Hope that makes sense?

coco98 commented 3 years ago

Is this affecting your CI/CD setup? That shouldn't be the case, so do let us know how that's being affected and we can help out!

If I understand correctly, you're looking for the "migrations" history table which you can't see anymore and that's causing a problem?

brandonbaraban commented 3 years ago

Hi, @coco98. I'm facing a similar problem to what you are saying. We recently switched from deploying our own production and staging backends using the Hasura docker image to using Hasura Cloud for both. Previously, changes in the console were tracked in a local migrations directory, and that is how we kept staging and production in sync and also tested out changes on staging before applying them to production. I don't see a way to do that through Hasura Cloud as it is now. Would appreciate any insights you have here!

brandonbaraban commented 3 years ago

I also don't see a way to use many of the CLI features on Hasura Cloud projects, like applying seeds. If there is any way to get that working, that would also be very useful.