404 pages return html in json response

[JanStevens]

Issue Summary

When a 404 gets triggered because of a dynamic page which does not exist, the returned json for that non existing page contains the HTML of the 404. This causes the frontend javascript code to crash resulting in a 500 error or a blank error page.

We notice that 404 pages cause javascript to crash since the retrieve page .json file results in a generated html file instead of a json response. Locally when we trigger a 404 we get back for the json response {notFound: true}. This only happens when we are working with dynamic routes (eg: [...slug].tsx).

Actual behavior

In development when triggering the same 404 page the following requests are made for the url http://localhost:3000/events/page-does-not-exist

1. GET /events/page-does-not-exist is called and returns the blocking (with spinner) HTML in our case.
2. GET /_next/data/development/en/events/page-does-not-exist.json is called and returns the json object {notFound: true} which causes next to request the 404 page json
3. GET /_next/data/development/en/404.json is called containing the pageProps and the 404 page is correctly displayed.

On our production environment for the same url we notice the following relevant requests:

1. GET /events/page-does-not-exist is called and returns the blocking (with spinner) HTML in our case.
2. GET /_next/data/9snNPN2gYA7NlY33qSWFq/en/events/page-does-not-exist.json is called and returns the 404 page as HTML, this causes a crash since nextjs expect it to be json

Expected behavior

When a page is not found we should return {notFound: true} instead of generating a 404 html page for a JSON response

Steps to reproduce

Have a custom 404 page with getStaticProps that passes some props to the 404 page. Example

type Props = Required<Awaited<ReturnType<typeof getStaticProps>>>['props'];
const NotFound: NextPage<Props> = ({ pageNotFound = {} }) => {
  usePageView();
  const { title, description, links = [] } = pageNotFound;
  return <Page404 title={title} description={description} links={links} />;
};

export default NotFound;

export const getStaticProps = async ({
  locale = DEFAULT_LOCALE,
}: GetStaticPropsContext) => {
  // Example data, we normally perform a fetch from our CMS
  const pageNotFound = { title: '404', description: 'Could not find the page, are you looking for:', links: ['https://google.com'] };

  return {
    props: {
      pageNotFound,
    },
    revalidate: Duration.MINUTE,
  };
};

Screenshots/Code/Configuration/Logs

Versions

• OS/Environment: Mac / Linux
• @sls-next/serverless-component version: 3.7.0
• Next.js version: 12.1.4

Additional context

serverless.yml

name: mdlbeast-frontend-nextjs-dev

mdlBeastFrontend:
  component: '@sls-next/serverless-component@3.7.0'
  inputs:
    useServerlessTraceTarget: true
    memory: 2048
    bucketName: frontend-nextjs-s3
    description: '*lambda-type*@Edge for frontend-nextjs'
    domain: dev.mdlbeast.com
    name:
      defaultLambda: frontend-nextjs-lambda-default
      apiLambda: frontend-nextjs-lambda-api
      imageLambda: frontend-nextjs-lambda-image
    publicDirectoryCache:
      value: public, max-age=31536000, must-revalidate
      test: /\.(gif|jpe?g|jp2|tiff|png|webp|bmp|svg|ico|woff|woff2)$/i
    build:
      env:
        NODE_OPTIONS: '--max-old-space-size=4096'
    # Ensure API routes can access the Host param
    cloudfront:
      api/*:
        forward:
          headers:
            - Host
            - Referer

Checklist

• [x] You have reviewed the README and FAQs, which answers several common questions.
• [x] You have reviewed our DEBUGGING wiki and have tried your best to include complete information and reproduction steps (including your configuration) as is possible. As there is only one maintainer (who maintains this in his free time) and thus very limited resources, if you have time, please try to debug the issue a bit yourself if possible.
• [x] You have first tried using the most recent latest or alpha @sls-next/serverless-component release version, which may have already fixed your issue or implemented the feature you are trying to use. Note that the old serverless-next.js component and the serverless-next.js plugin are deprecated and no longer maintained.

[nick-verstocken]

@dphang Is there any updates on this? This is causing a lot of false/positives on our prod environment

[alicercedigital]

Have the same problem.

[fasani-tx]

+1

Any news @JanStevens ?

[sureshHARDIYA]

I have the same issue.

[johnrappelepam]

We have experienced the same issue. The steps to reproduce are slightly different but generally follow the same principle.

1. Run a build on Vercel
2. Watch for when Vercel is getting Static paths
3. As soon as it finishes getting static paths, REMOVE/UNPUBLISH one of the items it expects to find
4. Wait for the build to complete
5. After the build completes, navigate to the Source > Output and find the 404 JSON under /next/data//en/my/path.json
6. The contents of path.json will be the html of the 404 page

Now that the build is complete, the page in question will return a 404 correctly. BUT now REPUBLISH the page and then REVALIDATE it via on-demand ISR. This should "fix" the earlier issues and allow the page to be navigated, however, the behavior of the OP is present. The page is seen briefly until the request is made for the JSON of the page, at which point the page immediately switches to the custom error page.

In other words, once a page is in this state, it cannot be fixed until a new deployment occurs.

We leverage a CMS so content is a bit more volatile- which increases the likelihood of our scenario occurring.

A workaround we explored is to check for process.env.CI === '1' and then fail the build if the request fo the page content fails, but this isn't an ideal scenario as it should be allowed that pages can be published/unpublished whenever.