Fetching metadata takes a really long time

[osdiab]

Tried out this action, powering it with Neon branches instead of having the action make fresh databases; and it seems to take 4+ minutes to refetch the metadata (and also creates weird logs like in #57 ), which makes it a very slow and not-confidence-inducing preview app experience. Hope that this project gets built out better as per #132

[osdiab]

OK, much later I'm trying to debug this on a repo that contains a miniscule amount of metadata and a tiny migration; and this is what I'm finding from this log of a failed run (because i've set up the database wrong based on the logs) including some extra debugging logs:

• the github action basically posts a job to some Hasura internal job API, and so you don't really have any real control over what it's doing as a consumer of preview apps. this is a bummer.
• When it's logging, it's just polling that API every approx 2 seconds and printing a log based on the result
• the null logs are caused by job names that this github action just doesn't have coded in, and instead of just saying what the internal job name is, it says helpfully says null instead.
• Fetching Metadata doesn't mean fetching Hasura metadata, it means that a gh-validation task is running (so presumably this is something about introspecting the Github PR, perhaps ultimately to pull the metadata files from it). it is mysterious why this ends up taking around 2 minutes to resolve with relatively tiny metadata, which is the lions share of the runtime.
• parse-metadata is the next state (but it shows up as null to keep you on your toes), which resolved relatively quickly for me.
• next up is apply-metadata, this passed in like 20 seconds.
• Next up, another null log that turns out to be parse-migration. For a single really short migration, this took about 16 seconds
• then it applies the migrations

so yeah what it's doing seems to be mostly valid. gotta debug for myself why the database details aren't being injected properly from secrets. no idea why the gh-validation task takes so long, since i don't think we have any visibility into what it's doing, since it's something internal to Hasura Cloud. Hope this helps someone at Hasura prioritize this, as slow preview app spinup times + unhelpful logs are deal breakers for actually using this in a meaningful way. @wawhal @kikkupico @sfarqu

[osdiab]

OK, I got it to run through successfully. In total took about 7 mins 30 seconds for a stub app. So yeah, pretty crazy slow. Might be better off just using the Hasura Cloud API to spin everything up myself instead of using the preview apps APIs, or just not even bothering with Hasura Cloud for this and instead just creating it with free Render instances and the Hasura Community Edition Docker Image. I'll post later with some timestamps to see roughly how long each step is taking (but will be based on the 2 second polling time, so rounded to the next 2s)

[osdiab]

OK, here is the ultimate timestamps:

{
  "taskStartTimes": {
    "gh-validation: Fetching Metadata": "2023-06-18T13:36:40.842Z",
    "parse-metadata: unknown task: parse-metadata": "2023-06-18T13:39:09.078Z",
    "apply-metadata: Applying metadata": "2023-06-18T13:39:28.954Z",
    "parse-migration: unknown task: parse-migration": "2023-06-18T13:39:54.348Z",
    "apply-migration: Applying migrations": "2023-06-18T13:40:11.634Z",
    "reload-metadata: Refreshing metadata": "2023-06-18T13:40:42.042Z",
    "check-healthz: Checking Project Health": "2023-06-18T13:41:06.407Z"
  },
  "completionTime": "2023-06-18T13:43:37.940Z"
}

so

• parse-metadata and parse-migration lack names in the translation function, so they show up as null in the logs. maybe we should just show the actual internal name if a user friendly name hasn't been provided instead of just showing null in the logs.
• gh-validation is definitely the most egregious part, but check-healthz seems to take forever as well.
• from parse-metadata to reload-metadata, it's reasonable, but i think just running hasura deploy myself, or using the docker image that runs migrations and metadata automatically, would probably go faster.

So yeah, between this and #114 i think i'm probably just going to avoid using Hasura Cloud Preview Apps for now and instead do my own community edition deploys until Hasura actually gets the chance to spend some time making this experience polished.

[GMkonan]

Hey @osdiab have you come across another way to do this? Thinking about doing exactly what you did, using neon.tech branches for the databases and this action to power up hasura, but got kinda worried about those timestamps... Did you manage another solution?

[osdiab]

I just stopped using Hasura’s preview apps and instead am using their GraphQL API to spin up/down normal Hasura instances. This code is a bit outdated with some fixes I’ve done in a private repo but it gets the idea across:

https://github.com/osdiab/personal-website/tree/71ed261df88b33d3ec5325ee424601820f67cf8b/.github/workflows

Note that they have a 3 free tier project limit, but also note that their new professional instance pricing is only by active time fielding requests/running triggers etc - so even if you use them it’s not likely to be bad unless you have frequent scheduled triggers or something.