Hasura startup error through Docker after upgrade to latest image

[subodh022]

I had hasura image in dockerfile set to image: hasura/graphql-engine:latest and something happened today which broke the hasura and I am getting this error now - relation \"hdb_catalog.hdb_table\" does not exist

I tried to downgrade to v1.3.3 but get the same error. How do I fix this?

[gengue]

Same here. It seems during the upgrade many hdb_* tables were deleted, I don't know what to do

[tirumaraiselvan]

hasura/graphql-engine:latest will now be pointing to hasura/graphql-engine:v2.0.0-alpha.2

The upgrade should be seamless, when exactly are you getting the error relation \"hdb_catalog.hdb_table\" does not exist ?

[subodh022]

@tirumaraiselvan I got this error on 26th Feb, so I think it was after the 2.0 release. Also, does hasura/graphql-engine:latest update the version automatically? Because it was running on 1.3.3 earlier and these tables got deleted without any manual update.

I can see that I only have these tables in the database now -

I saw this error in docker logs -

{
  "internal": {
    "statement": "select\n  json_build_object(\n    'tables', tables.items :: json,\n    'relations', relations.items,\n    'permissions', permissions.items,\n    'event_triggers', event_triggers.items,\n    'remote_schemas', remote_schemas.items,\n    'functions', functions.items,\n    'allowlist_collections', allowlist.item,\n    'computed_fields', computed_field.items,\n    'custom_types', custom_types.item,\n    'actions', actions.items,\n    'remote_relationships', remote_relationships.items,\n    'cron_triggers', cron_triggers.items\n  )\nfrom\n  (\n    select\n      coalesce(jsonb_agg(\n        jsonb_build_object(\n          'name', jsonb_build_object(\n            'name', ht.table_name,\n            'schema', ht.table_schema\n          ),\n          'is_enum', ht.is_enum,\n          'is_system_defined', ht.is_system_defined,\n          'configuration', ht.configuration,\n          'info', t.info\n        )\n      ), '[]') as items\n    from hdb_catalog.hdb_table ht\n    left join hdb_catalog.hdb_table_info_agg t using (table_schema, table_name)\n  ) as tables,\n  (\n    select\n      coalesce(\n        json_agg(\n          json_build_object(\n            'table',\n            json_build_object(\n              'schema', table_schema,\n              'name', table_name\n            ),\n            'rel_name', rel_name,\n            'rel_type', rel_type,\n            'def', rel_def :: json,\n            'comment', comment\n          )\n        ),\n        '[]'\n      ) as items\n    from\n      hdb_catalog.hdb_relationship\n  ) as relations,\n  (\n    select\n      coalesce(\n        json_agg(\n          json_build_object(\n            'table',\n            json_build_object(\n              'schema', table_schema,\n              'name', table_name\n            ),\n            'role', role_name,\n            'perm_type', perm_type,\n            'def', perm_def :: json,\n            'comment', comment\n          )\n        ),\n        '[]'\n      ) as items\n    from\n      hdb_catalog.hdb_permission\n  ) as permissions,\n  (\n    select\n      coalesce(\n        json_agg(\n          json_build_object(\n            'table',\n            json_build_object(\n              'schema', schema_name,\n              'name', table_name\n            ),\n            'name', name,\n            'def', configuration :: json\n          )\n        ),\n        '[]'\n      ) as items\n    from\n      hdb_catalog.event_triggers\n  ) as event_triggers,\n  (\n    select\n      coalesce(\n        json_agg(\n          json_build_object(\n            'name',\n            name,\n            'definition', definition :: json,\n            'comment', comment\n          )\n        ),\n        '[]'\n      ) as items\n    from\n      hdb_catalog.remote_schemas\n  ) as remote_schemas,\n  (\n    select\n      coalesce(json_agg(q.info), '[]') as items\n    from\n        (\n        select\n          json_build_object(\n            'function',\n            json_build_object(\n              'schema', hf.function_schema,\n              'name', hf.function_name\n            ),\n            'configuration', hf.configuration,\n            'is_system_defined', hf.is_system_defined,\n            'info', hf_agg.function_info\n          ) as info\n        from\n          hdb_catalog.hdb_function hf\n        left join lateral\n            (\n              select coalesce(json_agg(function_info), '[]') as function_info\n              from hdb_catalog.hdb_function_info_agg\n               where function_name = hf.function_name\n                     and function_schema = hf.function_schema\n            ) hf_agg on 'true'\n      ) as q\n   ) as functions,\n  (\n    select\n      coalesce(json_agg(hqc.collection_defn), '[]') as item\n    from hdb_catalog.hdb_allowlist ha\n    left outer join\n         hdb_catalog.hdb_query_collection hqc\n         on (hqc.collection_name = ha.collection_name)\n  ) as allowlist,\n  (\n    select\n      coalesce(json_agg(\n        json_build_object('computed_field', cc.computed_field,\n                          'function_info', fi.function_info\n                         )\n      ), '[]') as items\n    from\n      (\n        select json_build_object(\n          'table', jsonb_build_object('name', hcc.table_name,'schema', hcc.table_schema),\n          'name', hcc.computed_field_name,\n          'definition', hcc.definition,\n          'comment', hcc.comment\n        ) as computed_field,\n        hccf.function_name,\n        hccf.function_schema\n        from hdb_catalog.hdb_computed_field hcc\n        left outer join\n             hdb_catalog.hdb_computed_field_function hccf\n             on ( hcc.table_name = hccf.table_name\n                 and hcc.table_schema = hccf.table_schema\n                 and hcc.computed_field_name = hccf.computed_field_name\n                )\n      ) cc\n    left join lateral\n      (\n        select coalesce(json_agg(function_info), '[]') as function_info\n        from hdb_catalog.hdb_function_info_agg\n        where function_name = cc.function_name and function_schema = cc.function_schema\n      ) fi on 'true'\n  ) as computed_field,\n  (\n    select\n      json_build_object(\n        'custom_types',\n         coalesce((select custom_types from hdb_catalog.hdb_custom_types), '{}'),\n        'pg_scalars', -- See Note [Postgres scalars in custom types]\n         coalesce((select json_agg(typname) from pg_catalog.pg_type where typtype = 'b'), '[]')\n      ) as item\n  ) as custom_types,\n  (\n    select\n      coalesce(\n        json_agg(\n          json_build_object(\n            'name', ha.action_name,\n            'definition', ha.action_defn :: json,\n            'comment', ha.comment,\n            'permissions', p.items\n          )\n        ),\n        '[]'\n      ) as items\n    from\n      hdb_catalog.hdb_action ha\n      left join lateral\n      (\n        select\n          coalesce(\n            json_agg(\n              json_build_object(\n                'action', hap.action_name,\n                'role', hap.role_name,\n                'comment', hap.comment\n              )\n            ),\n            '[]'\n          ) as items\n          from\n              hdb_catalog.hdb_action_permission hap\n          where hap.action_name = ha.action_name\n      ) p on 'true'\n  ) as actions,\n  (\n    select coalesce(json_agg(\n      json_build_object(\n        'name', remote_relationship_name,\n        'table', json_build_object('schema', table_schema, 'name', table_name),\n        'hasura_fields', definition -> 'hasura_fields',\n        'remote_schema', definition -> 'remote_schema',\n        'remote_field', definition -> 'remote_field'\n      )\n    ),'[]') as items\n    from hdb_catalog.hdb_remote_relationship\n  ) as remote_relationships,\n  (\n    select\n      coalesce(\n        json_agg(\n          json_build_object(\n            'name', name,\n            'webhook_conf', webhook_conf :: json,\n            'cron_schedule', cron_schedule,\n            'payload', payload :: json,\n            'retry_conf', retry_conf :: json,\n            'header_conf', header_conf :: json,\n            'comment', comment\n          )\n        ),\n        '[]'\n      ) as items\n      from\n          hdb_catalog.hdb_cron_triggers\n  ) as cron_triggers\n",
    "prepared": true,
    "error": {
      "exec_status": "FatalError",
      "hint": null,
      "message": "relation \"hdb_catalog.hdb_table\" does not exist",
      "status_code": "42P01",
      "description": null
    },
    "arguments": []
  },
  "path": "$",
  "error": "database query error",
  "code": "unexpected"
}

[tirumaraiselvan]

Yes, hasura/graphql-engine:latest will point to the latest release at any point of time and hence it is NOT recommended to use docker images this way. You should clearly specify the version you need unless you are sure you want to get auto-upgraded.

Because it was running on 1.3.3 earlier and these tables got deleted without any manual update.

v2.0 has a different catalog and it is expected to drop those tables.

Are you still getting this error with v2.0?

[amnaveenriaz]

@tirumaraiselvan Yes. I am also facing the same issue after upgrading to V2.0 in Heroku.

My other projects have no issues. Only 1 project seems to have this issue after upgrade. Downgrade does not fix it. Is there a workaround after downgrading to V1.3.3?

[subodh022]

@tirumaraiselvan I restored the database and reverted to the Hasura version 1.3.3. Will plan a proper upgrade after sometime.

[amnaveenriaz]

@tirumaraiselvan Upgrading is the issue. Recreating the DB from older DB worked even with 2.0

[kedarguy]

Did anyone manage to downgrade? I wanted to try hasura cloud, connected it to my dev database on heroku and now the heroku APP is crashing.

This is the log

{"internal":{"statement":"select\n json_build_object(\n 'tables', tables.items :: json,\n 'relations', relations.items,\n 'permissions', permissions.items,\n 'event_triggers', event_triggers.items,\n 'remote_schemas', remote_schemas.items,\n 'functions', functions.items,\n 'allowlist_collections', allowlist.item,\n 'computed_fields', computed_field.items,\n 'custom_types', custom_types.item,\n 'actions', actions.items,\n 'remote_relationships', remote_relationships.items,\n 'cron_triggers', cron_triggers.items\n )\nfrom\n (\n select\n coalesce(jsonb_agg(\n jsonb_build_object(\n 'name', jsonb_build_object(\n 'name', ht.table_name,\n 'schema', ht.table_schema\n ),\n 'is_enum', ht.is_enum,\n 'is_system_defined', ht.is_system_defined,\n 'configuration', ht.configuration,\n 'info', t.info\n )\n ), '[]') as items\n from hdb_catalog.hdb_table ht\n left join hdb_catalog.hdb_table_info_agg t using (table_schema, table_name)\n ) as tables,\n (\n select\n coalesce(\n json_agg(\n json_build_object(\n 'table',\n json_build_object(\n 'schema', table_schema,\n 'name', table_name\n ),\n 'rel_name', rel_name,\n 'rel_type', rel_type,\n 'def', rel_def :: json,\n 'comment', comment\n )\n ),\n '[]'\n ) as items\n from\n hdb_catalog.hdb_relationship\n ) as relations,\n (\n select\n coalesce(\n json_agg(\n json_build_object(\n 'table',\n json_build_object(\n 'schema', table_schema,\n 'name', table_name\n ),\n 'role', role_name,\n 'perm_type', perm_type,\n 'def', perm_def :: json,\n 'comment', comment\n )\n ),\n '[]'\n ) as items\n from\n hdb_catalog.hdb_permission\n ) as permissions,\n (\n select\n coalesce(\n json_agg(\n json_build_object(\n 'table',\n json_build_object(\n 'schema', schema_name,\n 'name', table_name\n ),\n 'name', name,\n 'def', configuration :: json\n )\n ),\n '[]'\n ) as items\n from\n hdb_catalog.event_triggers\n ) as event_triggers,\n (\n select\n coalesce(\n json_agg(\n json_build_object(\n 'name',\n name,\n 'definition', definition :: json,\n 'comment', comment\n )\n ),\n '[]'\n ) as items\n from\n hdb_catalog.remote_schemas\n ) as remote_schemas,\n (\n select\n coalesce(json_agg(q.info), '[]') as items\n from\n (\n select\n json_build_object(\n 'function',\n json_build_object(\n 'schema', hf.function_schema,\n 'name', hf.function_name\n ),\n 'configuration', hf.configuration,\n 'is_system_defined', hf.is_system_defined,\n 'info', hf_agg.function_info\n ) as info\n from\n hdb_catalog.hdb_function hf\n left join lateral\n (\n select coalesce(json_agg(function_info), '[]') as function_info\n from hdb_catalog.hdb_function_info_agg\n where function_name = hf.function_name\n and function_schema = hf.function_schema\n ) hf_agg on 'true'\n ) as q\n ) as functions,\n (\n select\n coalesce(json_agg(hqc.collection_defn), '[]') as item\n from hdb_catalog.hdb_allowlist ha\n left outer join\n hdb_catalog.hdb_query_collection hqc\n on (hqc.collection_name = ha.collection_name)\n ) as allowlist,\n (\n select\n coalesce(json_agg(\n json_build_object('computed_field', cc.computed_field,\n 'function_info', fi.function_info\n )\n ), '[]') as items\n from\n (\n select json_build_object(\n 'table', jsonb_build_object('name', hcc.table_name,'schema', hcc.table_schema),\n 'name', hcc.computed_field_name,\n 'definition', hcc.definition,\n 'comment', hcc.comment\n ) as computed_field,\n hccf.function_name,\n hccf.function_schema\n from hdb_catalog.hdb_computed_field hcc\n left outer join\n hdb_catalog.hdb_computed_field_function hccf\n on ( hcc.table_name = hccf.table_name\n and hcc.table_schema = hccf.table_schema\n and hcc.computed_field_name = hccf.computed_field_name\n )\n ) cc\n left join lateral\n (\n select coalesce(json_agg(function_info), '[]') as function_info\n from hdb_catalog.hdb_function_info_agg\n where function_name = cc.function_name and function_schema = cc.function_schema\n ) fi on 'true'\n ) as computed_field,\n (\n select\n json_build_object(\n 'custom_types',\n coalesce((select custom_types from hdb_catalog.hdb_custom_types), '{}'),\n 'pg_scalars', -- See Note [Postgres scalars in custom types]\n coalesce((select json_agg(typname) from pg_catalog.pg_type where typtype = 'b'), '[]')\n ) as item\n ) as custom_types,\n (\n select\n coalesce(\n json_agg(\n json_build_object(\n 'name', ha.action_name,\n 'definition', ha.action_defn :: json,\n 'comment', ha.comment,\n 'permissions', p.items\n )\n ),\n '[]'\n ) as items\n from\n hdb_catalog.hdb_action ha\n left join lateral\n (\n select\n coalesce(\n json_agg(\n json_build_object(\n 'action', hap.action_name,\n 'role', hap.role_name,\n 'comment', hap.comment\n )\n ),\n '[]'\n ) as items\n from\n hdb_catalog.hdb_action_permission hap\n where hap.action_name = ha.action_name\n ) p on 'true'\n ) as actions,\n (\n select coalesce(json_agg(\n json_build_object(\n 'name', remote_relationship_name,\n 'table', json_build_object('schema', table_schema, 'name', table_name),\n 'hasura_fields', definition -> 'hasura_fields',\n 'remote_schema', definition -> 'remote_schema',\n 'remote_field', definition -> 'remote_field'\n )\n ),'[]') as items\n from hdb_catalog.hdb_remote_relationship\n ) as remote_relationships,\n (\n select\n coalesce(\n json_agg(\n json_build_object(\n 'name', name,\n 'webhook_conf', webhook_conf :: json,\n 'cron_schedule', cron_schedule,\n 'payload', payload :: json,\n 'retry_conf', retry_conf :: json,\n 'header_conf', header_conf :: json,\n 'comment', comment\n )\n ),\n '[]'\n ) as items\n from\n hdb_catalog.hdb_cron_triggers\n ) as cron_triggers\n","prepared":true,"error":{"exec_status":"FatalError","hint":null,"message":"relation \"hdb_catalog.hdb_table\" does not exist","status_code":"42P01","description":null},"arguments":[]},"path":"$","error":"database query error","code":"unexpected"}