Schema initialization failure/false start (v 1.1.0)

scribahti commented 4 years ago

I collapsed the migration files as instructed here to create a set of "init" migration files. (I also have a freshly exported metadata.yaml file.)

$ tree migrations/
migrations/
├── 1584481060641_init
│   ├── up.sql
│   └── up.yaml
└── metadata.yaml

Next, I attempted applying this migration to a fresh Hasura deployment as follows:

$ hasura migrate apply --version 1584481060641 --endpoint {url}

On the first attempt, I received a FATA[0034] response as follows:

FATA[0034] apply failed: failed parsing json: invalid character '<' looking for beginning of value; response from API:
<!DOCTYPE html>
    <html>
      <head>
        <meta name="viewport" content="width=device-width, initial-scale=1">
        <meta charset="utf-8">
        <title>Application Error</title>
        <style media="screen">
          html,body,iframe {
            margin: 0;
            padding: 0;
          }
          html,body {
            height: 100%;
            overflow: hidden;
          }
          iframe {
            width: 100%;
            height: 100%;
            border: 0;
          }
        </style>
      </head>
      <body>
        <iframe src="//www.herokucdn.com/error-pages/application-error.html"></iframe>
      </body>
    </html>

On a subsequent attempt I receive the following:

FATA[0003] apply failed: Version already applied in database

Visually examining the console, it appears that my schemas are in place, but this is not a rigorous audit. Is this a known issue related to #3983 or #4117?

tirumaraiselvan commented 4 years ago

@scribahti FATA[0034] apply failed: failed parsing json: invalid character '<' looking for beginning of value; response from API:

You may note from the error response (which is in HTML) that Heroku is throwing an Application Error which happens when Hasura is not running. Logs of Hasura when this happens are worth checking.

Apart from this, FATA[0003] apply failed: Version already applied in database means that the migrations are already applied. Can you try applying this to a different/new instance and see if the error comes again.

Otherwise, we will need some kind of repro to discern the issue here because it is not clear.

scribahti commented 4 years ago

@tirumaraiselvan thanks for your note. I applied the migration to another fresh deployment with the same result. I need to review to review the logs & will comment here shortly.

scribahti commented 4 years ago

These instances are deployed via Heroku, but I am unable to log in to the vm.

$ heroku ps:exec -a {app}
Establishing credentials... error
 ▸    Could not connect to dyno!
 ▸    Check if the dyno is running with `heroku ps'

scribahti commented 4 years ago

@tirumaraiselvan how should I proceed to find logs?

tirumaraiselvan commented 4 years ago

@scribahti You need not exec into the VM. You can get application logs via the heroku logs command: https://devcenter.heroku.com/articles/logging#view-logs

scribahti commented 4 years ago

@tirumaraiselvan looks like a timeout with H12 error code.

2020-03-23T22:45:04.253443+00:00 heroku[router]: at=error code=H12 desc="Request timeout" method=POST path="/v1/query" host={url} request_id={id} fwd="{ip}" dyno=web.1 connect=0ms service=30337ms status=503 bytes=0 protocol=https

scribahti commented 4 years ago

@tirumaraiselvan any further insight on this?

hasura / graphql-engine

Schema initialization failure/false start (v 1.1.0) #4134