Failing to enqueue dynamically-created URLs

[TimTinkers]

Describe the bug

Hi @adamhenson thanks for the great work on this action.

I am using the excellent Wrangler Action to automatically deploy my pull requests as previews on Cloudflare Workers. As part of this, my workflow will deploy my site to a dynamic URL that I can intercept as <branch name>.company.workers.dev. When attempting to use my Foo API token to run Lighthouse against this site (on the branch lighthouse), I am experiencing the following error:

Run foo-software/lighthouse-check-action@master
  with:
    fooApiToken: ***
    urlsJson: [[ "***", "lighthouse.company.workers.dev" ]]
  env:
    BRANCH_NAME: lighthouse
lighthouse-check: Enqueueing Lighthouse audits.
lighthouse-check: All URLs failed to be enqueued.
lighthouse-check: Below is the API response for attempted URLs as an array. [
  { code: 'ERROR_GENERIC', message: 'Sorry, something went wrong.' },
  {
    code: 'ERROR_PAGE_MISSING',
    message: 'This page does not exist.',
    status: 422
  }
]
Error: All URLs failed to be enqueued.

To Reproduce

Set up a new free Foo account and attempt to monitor a Cloudflare Workers site being deployed via the Wrangler Action.

Expected behavior

I would expect the lighthouse.company.workers.dev site to be evaluated by Lighthouse with the results stored to Foo.

Additional context

This is the GitHub workflow that I am currently using.

name: Lighthouse
on: [ pull_request ]

jobs:
  lighthouse-check:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@master
      - name: Prepare environment
        run: |
          echo "BRANCH_NAME=$(echo "${{ github.head_ref || github.ref }}" | sed s'/refs\/heads\///g' | sed s'/[/.]/-/g' | tr '[:upper:]' '[:lower:]')" >> ${GITHUB_ENV}
      - name: Install dependencies
        run: npm install
      - name: Run Lighthouse
        uses: foo-software/lighthouse-check-action@master
        with:
          fooApiToken: ${{ secrets.FOO_API_SECRET }}
          urlsJson: '[[ "${{ secrets.FOO_API_SECRET }}", "${{ env.BRANCH_NAME }}.company.workers.dev" ]]'

Thank you!

[adamhenson]

Hi @TimTinkers - thank you for providing the details! You're on the right track and sorry that parts of the documentation aren't clear. I will improve this part of the config section and also I noticed a broken link in the docs... so, I'm sure that didn't help.

I see 2 things, but the error is coming from the first.

1. You'll actually need 2 different types of tokens here... the API token, which appears to have correctly configured by you, but also the page token. I think this part needs to be improved in the docs.

So, this line should have the page token. In the example below, I named the secret FOO_PAGE_TOKEN_SECRET, but that can obviously be anything... or hard-coded.

- urlsJson: '[[ "${{ secrets.FOO_API_SECRET }}", "${{ env.BRANCH_NAME }}.company.workers.dev" ]]'
+ urlsJson: '[[ "${{ secrets.FOO_PAGE_TOKEN_SECRET }}", "${{ env.BRANCH_NAME }}.company.workers.dev" ]]'

You're probably wondering where the heck you get that, and again - I think it's missing from the docs. You can get it from the page dashboard on Foo. Example below from the demo account.

2. Once you have corrected the above, I would imagine another error to be thrown... and I think the fix is simple here. lighthouse.company.workers.dev isn't a fully qualified URL, so I think the fix is to just prefix it with http:// or https://

Let me know if this works for you and / or close the issue if it does. Otherwise feel free to comment with further issues or questions regarding this.

[TimTinkers]

@adamhenson thanks for the reply and for catching that URL issue for me! That all makes sense.

Regarding the page token... how can I get that without adding it to the dashboard? I envision this being part of our CI where these pages are being created dynamically for every branch. I don't know their names ahead of time. All I know is that they are a subdomain of .company.workers.dev.

Can I get a single fixed Page API token with wildcard support? i.e. would the token for https://*.company.workers.dev be valid to use here?

It's very similar to the Vercel example from the docs:

name: Lighthouse
on: [pull_request]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: amondnet/vercel-action@v20
        with:
          vercel-token: ${{ secrets.VERCEL_TOKEN }} # Required
          github-token: ${{ secrets.GITHUB_TOKEN }} #Optional 
          vercel-args: '--prod' #Optional
          vercel-org-id: ${{ secrets.ORG_ID}}  #Required
          vercel-project-id: ${{ secrets.PROJECT_ID}} #Required 
          working-directory: ./sub-directory
      - name: Lighthouse
        uses: foo-software/lighthouse-check-action@master
        with:
          fooApiToken: ${{ secrets.LIGHTHOUSE_CHECK_API_TOKEN }}
          urlsJson: '[["fooPageToken", "${{ steps.now.outputs.preview-url }}"]]'

Except here I'm confused because it sounds like the Vercel example also wouldn't even work because fooPageToken would be completely dynamic and different depending on the value of steps.now.outputs.preview-url, no?

[adamhenson]

Hi @TimTinkers - This GitHub Action can be used in 2 different, separate ways. The 1st way is to run it on GitHub's runners without any association of Foo in which you run it on URLs you specify. The 2nd way is to trigger Lighthouse audits on "pages" you've added on Foo.

Each page you add on Foo is a unique URL to run Lighthouse against. For every page you add, a unique "page token" is generated and in this context, it's how we identify it. With free accounts, you can add up to 5 pages. Foo allows you to build a historical timeline of Lighthouse results for pages you add. With that said - I'm not following why you would want a different page for every PR.

I'm not quite understanding the ask here... perhaps you could elaborate.

[adamhenson]

Let me try to better explain how one may want to use this GitHub Action with Foo.

Let's say I add the following "pages" on Foo:

Page 1 URL: https://www.foo.software/posts Page API Token: abc Name: posts

Page 2 URL: https://www.foo.software/pricing Page API Token: def Name: pricing

Example Case 1: Post Deploy Lighthouse Audits

The standard case for triggering Lighthouse audits on pages that have been added on Foo would probably be like a post deploy step and it would look like so:

urls: 'abc,def'

So maybe you have one step in a workflow that deploys to production and another step that runs Lighthouse via this GitHub Action against URLs you've added on Foo with the above line in your workflow file.

Example Case 2: Pre Deploy Lighthouse Audit

There may be another case in which you want to run Lighthouse on pages you've added on Foo before they reach production. You may want to test them on different branches of code changes to see their impact... by the PR.

In this case you might have ephemeral URLs via Vercel or Cloudflare or whatever provider. In this case the domain will change, but the paths should still correspond to what you've added on Foo.

Page 1 URL: https://some-temp-url/posts

Page 2 URL: https://some-temp-url/pricing

In this case above your configuration would mirror the Vercel example like so.

urlsJson: '[["abc", "${{ steps.something.output }}/posts"], ["def", "${{ steps.something.output }}/pricing"]]'

[TimTinkers]

Hi @adamhenson, your "Example Case 2" is close to describing what I want to do. Does the URL I specify on Foo for the page need to match what is created by the build tool? That is, when comparing

Page 1
URL: https://some-temp-url/posts

and urlsJson: '[["abc", "${{ steps.something.output }}/posts"] ..., must ${{ steps.something.output }} equal https://some-temp-url? If so, then this is not a viable solution for me because I will not know the value of ${{ steps.something.output }} until the action runs and therefore cannot go into Foo to create that page.

[adamhenson]

No @TimTinkers the URL doesn’t need to match what you add on Foo, but it needs represent the same page. This page might have a different URL as a lower environment for example. So in my example I might use the production URL for the pricing and posts page when I add them on Foo, but with this solution I can run the GitHub Action against any URL as long as I identify it with the corresponding page token.

In case 1, the URL is not "dynamic". We want to audit the production URL exactly how it was entered in Foo for both the "posts" page and "pricing" page:

urls: 'abc,def'

In case 2, the URL is "dynamic". We want to audit a lower environment URL (an ephemeral URL from a PR) generated by the hosting provider for both the "posts" page and "pricing" page... and we would have a line that looks like this:

urlsJson: '[["abc", "${{ steps.something.output }}/posts"], ["def", "${{ steps.something.output }}/pricing"]]'

In the above the results are sent to Foo along with the page token which is what Foo uses to store them and associate them with a page added on Foo.

Is there still an issue? Were you able to get this working?

[adamhenson]

Closing as the source of the issue seems to have been identified and addressed. @TimTinkers - please let me know if I'm off or you have further questions.