[Bug]: next.js _middleware.js NextResponse.rewrite() returns 404, but works in netlify-cli

christopherpickering commented 2 years ago

Summary

Hi, We have a next.js middleware function that serves a static file, and allows us to rename the file with an environment variable.

the original file is /umami.js, and with a ENV variable of "TRACKER_SCRIPT_NAME=something_cool", I need to access the same file with the url /something_cool.js.

The middleware works fine with running with netlify-cli locally, but when deployed to netlify, I get a 404 error on the custom url. I added console logging to the middleware and everything looks as-expected.

Steps to reproduce

To run locally: create a .env file with

DATABASE_URL=postgresql://<path to heroku db>
HASH_SALT=super-secret-hash
TRACKER_SCRIPT_NAME=something_cool

initialize database
Start the app with netlify dev, from terminal.
Access localhost:888/umami.js - works
Access localhost:888/something_cool.js - works - same thing you saw w/ prev step.
Deploy to netlify
Access /umami.js - works
Access /something_cool.js - gives a 404. function logs look fine. Browser headers looks like it tried to send the file, but it gave a 404.

Here's the umami middleware:

import { NextResponse } from 'next/server';

function redirectHTTPS(req) {
  if (
    process.env.FORCE_SSL &&
    !req.headers.get('host').includes('localhost') &&
    req.nextUrl.protocol !== 'https'
  ) {
    return NextResponse.redirect(`https://${req.headers.get('host')}${req.nextUrl.pathname}`, 301);
  }
}

function customScriptName(req) {
  const scriptName = process.env.TRACKER_SCRIPT_NAME;
  console.log(scriptName);
  if (scriptName) {
    const url = req.nextUrl.clone();
    const { pathname } = url;
    console.log(url, pathname);

    if (pathname.endsWith(`/${scriptName}.js`)) {
      url.pathname = '/umami.js';
      console.log(url);
      return NextResponse.rewrite(url);
    }
  }
}

function disableLogin(req) {
  if (process.env.DISABLE_LOGIN && req.nextUrl.pathname.endsWith('/login')) {
    return new Response('403 Forbidden', { status: 403 });
  }
}

export function middleware(req) {
  const fns = [redirectHTTPS, customScriptName, disableLogin];

  for (const fn of fns) {
    const res = fn(req);
    if (res) {
      return res;
    }
  }

  return NextResponse.next();
}

A link to a reproduction repository

https://github.com/atlas-bi/website-analytics

Plugin version

4.2.7

More information about your build

[X] I am building using the CLI
[ ] I am building using file-based configuration (netlify.toml)

What OS are you using?

Mac OS

Your netlify.toml file

`netlify.toml`

```toml [functions] included_files = ["public/geo/*.mmdb"] ```

Your public/_redirects file

`_redirects`

```toml # Paste content of your `_redirects` file here ```

Your `next.config.js` file

`next.config.js`

```toml require('dotenv').config(); const pkg = require('./package.json'); module.exports = { env: { VERSION: pkg.version, }, basePath: process.env.BASE_PATH, eslint: { ignoreDuringBuilds: true, }, webpack(config) { config.module.rules.push({ test: /\.svg$/, issuer: /\.js$/, use: ['@svgr/webpack'], }); return config; }, async headers() { return [ { source: `/(.*\\.js)`, headers: [ { key: 'Cache-Control', value: 'public, max-age=2592000', // 30 days }, ], }, ]; }, }; ```

Builds logs (or link to your logs)

Build logs

[Link to logs](https://app.netlify.com/sites/focused-banach-ee0636/deploys/62322686632058000960d42e)

Function logs

``` Mar 16, 01:07:15 PM: 27c14d7b i18n: null,Mar 16, 01:07:15 PM: 27c14d7b trailingSlash: falseMar 16, 01:07:15 PM: 27c14d7b },Mar 16, 01:07:15 PM: 27c14d7b basePath: ''Mar 16, 01:07:15 PM: 27c14d7b }Mar 16, 01:07:15 PM: 27c14d7b } /api/auth/verifyMar 16, 01:07:15 PM: 27c14d7b INFO [GET] /api/auth/verify (SSR)Mar 16, 01:07:15 PM: 140b67f6 INFO something_coolMar 16, 01:07:15 PM: 140b67f6 INFO NextURL {Mar 16, 01:07:15 PM: 140b67f6 [Symbol(NextURLInternal)]: {Mar 16, 01:07:15 PM: 140b67f6 url: URL {Mar 16, 01:07:15 PM: 140b67f6 href: 'http://62322686632058000960d42e--focused-banach-ee0636.netlify.app/api/websites',Mar 16, 01:07:15 PM: 140b67f6 origin: 'http://62322686632058000960d42e--focused-banach-ee0636.netlify.app',Mar 16, 01:07:15 PM: 140b67f6 protocol: 'http:',Mar 16, 01:07:15 PM: 140b67f6 username: '',Mar 16, 01:07:15 PM: 140b67f6 password: '',Mar 16, 01:07:15 PM: 140b67f6 host: '62322686632058000960d42e--focused-banach-ee0636.netlify.app',Mar 16, 01:07:15 PM: 140b67f6 hostname: '62322686632058000960d42e--focused-banach-ee0636.netlify.app',Mar 16, 01:07:15 PM: 140b67f6 port: '',Mar 16, 01:07:15 PM: 140b67f6 pathname: '/api/websites',Mar 16, 01:07:15 PM: 140b67f6 search: '',Mar 16, 01:07:15 PM: 140b67f6 searchParams: URLSearchParams {},Mar 16, 01:07:15 PM: 140b67f6 hash: ''Mar 16, 01:07:15 PM: 140b67f6 },Mar 16, 01:07:15 PM: 140b67f6 options: {Mar 16, 01:07:15 PM: 140b67f6 basePath: '',Mar 16, 01:07:15 PM: 140b67f6 headers: [Object],Mar 16, 01:07:15 PM: 140b67f6 i18n: null,Mar 16, 01:07:15 PM: 140b67f6 trailingSlash: falseMar 16, 01:07:15 PM: 140b67f6 },Mar 16, 01:07:15 PM: 140b67f6 basePath: ''Mar 16, 01:07:15 PM: 140b67f6 }Mar 16, 01:07:15 PM: 140b67f6 } /api/websitesMar 16, 01:07:15 PM: 140b67f6 INFO [GET] /api/websites (SSR)Mar 16, 01:07:15 PM: 27c14d7b Duration: 38.64 ms Memory Usage: 186 MB Mar 16, 01:07:15 PM: 140b67f6 Duration: 32.08 ms Memory Usage: 186 MB Mar 16, 01:07:15 PM: a2ecf142 INFO something_coolMar 16, 01:07:15 PM: a2ecf142 INFO NextURL {Mar 16, 01:07:15 PM: a2ecf142 [Symbol(NextURLInternal)]: {Mar 16, 01:07:15 PM: a2ecf142 url: URL {Mar 16, 01:07:15 PM: a2ecf142 href: 'http://62322686632058000960d42e--focused-banach-ee0636.netlify.app/login',Mar 16, 01:07:15 PM: a2ecf142 origin: 'http://62322686632058000960d42e--focused-banach-ee0636.netlify.app',Mar 16, 01:07:15 PM: a2ecf142 protocol: 'http:',Mar 16, 01:07:15 PM: a2ecf142 username: '',Mar 16, 01:07:15 PM: a2ecf142 password: '',Mar 16, 01:07:15 PM: a2ecf142 host: '62322686632058000960d42e--focused-banach-ee0636.netlify.app',Mar 16, 01:07:15 PM: a2ecf142 hostname: '62322686632058000960d42e--focused-banach-ee0636.netlify.app',Mar 16, 01:07:15 PM: a2ecf142 port: '',Mar 16, 01:07:15 PM: a2ecf142 pathname: '/login',Mar 16, 01:07:15 PM: a2ecf142 search: '',Mar 16, 01:07:15 PM: a2ecf142 searchParams: URLSearchParams {},Mar 16, 01:07:15 PM: a2ecf142 hash: ''Mar 16, 01:07:15 PM: a2ecf142 },Mar 16, 01:07:15 PM: a2ecf142 options: {Mar 16, 01:07:15 PM: a2ecf142 basePath: '',Mar 16, 01:07:15 PM: a2ecf142 headers: [Object],Mar 16, 01:07:15 PM: a2ecf142 i18n: null,Mar 16, 01:07:15 PM: a2ecf142 trailingSlash: falseMar 16, 01:07:15 PM: a2ecf142 },Mar 16, 01:07:15 PM: a2ecf142 basePath: ''Mar 16, 01:07:15 PM: a2ecf142 }Mar 16, 01:07:15 PM: a2ecf142 } /loginMar 16, 01:07:15 PM: a2ecf142 INFO [GET] /login (SSR)Mar 16, 01:07:15 PM: a2ecf142 Duration: 5.70 ms Memory Usage: 186 MB ```

.next JSON files

generated .next JSON files

``` # Paste file contents here. Please check there isn't any private info in them # You can either build locally, or download the deploy from Netlify by clicking the arrow next to the deploy time. ```

bjornallvin commented 2 years ago

Do you have password protection on your site on Netlify? That seems to be the issue in my case. Middleware does not seem to work with password protected sites on Netlify.

ascorbic commented 2 years ago

You are correct that Edge Functions do not currently work with password protection. If that's not the issue, rewrite problems may be related to this issue, which is fixed in the latest version of the plugin. @christopherpickering It would be great if you could try deploying again and see if it works now.

christopherpickering commented 2 years ago

Thanks for the comments! The function was not using password protection. I'll give it a retry.

christopherpickering commented 2 years ago

@ascorbic thanks, I redid the deploy: https://app.netlify.com/sites/focused-banach-ee0636/deploys/626807debad45243099b1eb0

It used: @netlify/plugin-nextjs@4.4.4 from Netlify app

I get the same response as before. Should I try anything else?

christopherpickering commented 2 years ago

Here's the related issue w/ more comments. This works fine in the netlify cli. https://github.com/mikecao/umami/issues/1008

ascorbic commented 2 years ago

Hey. Could you try enabling edge functions for middleware? You should set the environment variable NEXT_USE_NETLIFY_EDGE to true

christopherpickering commented 2 years ago

Hey @ascorbic thanks! Here's the build: https://app.netlify.com/sites/focused-banach-ee0636/deploys/6269472b35bed1324c5f6f36

I did a clear cache + build, Still no go.

MarcL commented 4 months ago

Hey @christopherpickering!

The team at Netlify have now released the new v5 Next.js runtime which has support for both pages and app router, on-demand and time-based revalidation, automatic fine-grained cache control, and automatic image optimization using Netlify's image CDN. Can you try this with the new runtime as it should be fixed?

You can find the documentation and additional information here: Next.js on Netlify

The v4 runtime is now in the maintenance support phase with no new features being added. Occasional bug fixes and security patches will be applied when needed.

Thanks!

netlify / next-runtime