Slower Response Times when updating from 10.0.7 to 10.2.0 or 11.0.1 or 12.0.1 (with Disabled automatic image optimization)

[fabb]

What version of Next.js are you using?

10.2.0, 10.2.1-canary.3, 11.0.1, 12.0.1

What version of Node.js are you using?

14.16.1

What browser are you using?

all

What operating system are you using?

all

How are you deploying your application?

own k8s cluster with custom express server

Describe the Bug

When updating from 10.0.7 to 10.2.0 or 10.2.1-canary.3 with webpack 4, response times and getInitialProps render times of real user traffic increase by 8-12%.

We are not using next.js automatic image optimization at all. Note that for the traffic response time we only measure the initial html request, not requests for _next assets and chunks.

The dotted vertical line marks the release. We are doing a rolling release with k8s, it took around 8 minutes until all new pods took over.

We are running 100 pods in parallel and have around 4k requests/min (pretty stable during the time in the graph), so the average reponse time graph should be reliable.

We also tried 10.2.1-canary.3 and 11.0.1 with webpack 5, but response times were even slightly worse than with webpack 4.

Expected Behavior

Render times should not increase when updating next.js.

To Reproduce

Unfortunately it's a closed-source project. If you need more detailed information about something, please let me know.

[timneutkens]

Unfortunately it's a closed-source project. If you need more detailed information about something, please let me know.

A reproduction would be helpful, we can't investigate based on this report.

[fabb]

Yeah, I understand that though I don‘t know where to start. Our project is huge and reducing it to something that still shows the issue but is open-source-able would be a lot of work i currently cannot invest. For the time being I just hope someone with an open source project has the same issue (maybe one of the upvoters?). I can offer to add debug statements to narrow down the cause (if you have suggestions), or to try new canary versions.

[karlhorky]

Based on the paths above, I guess this website is https://www.willhaben.at/iad ? Not sure whether this information is useful for coming up with a theory of what is making it slower, but at least it shows the general complexity of the pages...

[karlhorky]

reducing it to something that still shows the issue

Maybe instead of a reduced test case, what about listing out some things that not all applications would use:

1. custom Next.js configuration
2. what are you doing with the custom express server (on a granular level - eg. why can't you just use the normal Next.js next start to serve?)
3. use of Next.js integration points, such as _document.js, _app.js (and what you're generally doing there)
4. what type of static generation / server-side rendering are you using? any interesting features like "Incremental Static Regeneration"?

I guess since you are not using the Image component, it could also possibly be unrelated to the image change, but related to another change between those versions. Did you take a detailed look at the other changes in 10.0.8? (I'm assuming that this already caused problems with 10.0.8)

[goguda]

Unfortunately it's a closed-source project. If you need more detailed information about something, please let me know.

A reproduction would be helpful, we can't investigate based on this report.

Please see my comment here https://github.com/vercel/next.js/issues/24948#issuecomment-839428991 , most likely related to the same issue, if the debug url I've set up can be of any help.

[fabb]

Based on the paths above, I guess this website is https://www.willhaben.at/iad ? Not sure whether this information is useful for coming up with a theory of what is making it slower, but at least it shows the general complexity of the pages...

Well this exact url is served from a legacy system, but yes, this is the domain of our application. /iad/immobilien and /iad/gebrauchtwagen is served from our next.js application.

[fabb]

Maybe instead of a reduced test case, what about listing out some things that not all applications would use

Ok, I‘ll do a write-up in the next few days.

So much for now: we‘re mixing SSR (with getInitialProps) and SSG, but since SSG ist extremely fast anyways, we don‘t really see changes there. We don‘t use ISR. We use styled-components which could have an impact on SSR render time (collecting styles in _document).

[fabb]

Did you take a detailed look at the other changes in 10.0.8? (I'm assuming that this already caused problems with 10.0.8)

We tried updating to 10.0.9 some weeks ago, but our application ran into a crash cycle so we needed to roll back to 10.0.7.

https://github.com/vercel/next.js/issues/23189#issuecomment-802675596

[fabb]

Ok, so here we go.

Custom Next.js configuration:

• inject polyfills
• @sentry import alias
• TerserPlugin config
• next-plugin-custom-babel-config and next-transpile-modules (because of monorepo)
• bundle analyzer
• custom static import plugin
• some next.js configs
• next.config.js

  ``` const Webpack = require('webpack') const namespace = process.env.NAMESPACE const buildIdArgument = process.env.BUILD_ID const withStaticImport = require('./buildtools/plugin-static') const withBundleAnalyzer = require('@next/bundle-analyzer')({ enabled: process.env.ANALYZE === 'true', }) const path = require('path') const withTranspileModules = require('next-transpile-modules') const withPlugins = require('next-compose-plugins') const withCustomBabelConfigFile = require('next-plugin-custom-babel-config') function withCustomWebpack(config2 = {}) { const { webpack } = config2 config.webpack = (config, options) => { const { isServer, dev, buildId } = options const originalEntry = config.entry config.entry = async () => { const entries = await originalEntry() if (entries['main.js'] && !entries['main.js'].includes('./src/polyfills/polyfills.ts')) { entries['main.js'].unshift('./src/polyfills/polyfills.ts') } return entries } if (!dev) { config.plugins.push(new Webpack.DefinePlugin({ 'process.env.SENTRY_RELEASE': JSON.stringify(buildId) })) } // In `pages/_app.js`, Sentry is imported from @sentry/node. While // @sentry/browser will run in a Node.js environment, @sentry/node will use // Node.js-only APIs to catch even more unhandled exceptions. // // This works well when Next.js is SSRing your page on a server with // Node.js, but it is not what we want when your client-side bundle is being // executed by a browser. // // Luckily, Next.js will call this webpack function twice, once for the // server and once for the client. Read more: // https://nextjs.org/docs#customizing-webpack-config // // So ask Webpack to replace @sentry/node imports with @sentry/browser when // building the browser's bundle if (!isServer) { config.resolve.alias['@sentry/node'] = '@sentry/browser' } if (!dev) { if (config.optimization && config.optimization.minimizer) { for (const plugin of config.optimization.minimizer) { if (plugin.constructor.name === 'TerserPlugin') { plugin.options.sourceMap = true plugin.options.terserOptions = { ...plugin.options.terserOptions, warnings: false, compress: { ...plugin.options.terserOptions.compress, drop_console: !isServer, }, } break } } } } return webpack(config, options) } return config } const plugins = [ [ withCustomBabelConfigFile({ babelConfigFile: path.resolve('../../babel.config.js'), }), ], [withTranspileModules(['@wh'])], [withStaticImport], [withBundleAnalyzer], [withCustomWebpack], ] const config = { generateBuildId: async () => { return buildIdArgument || 'local' }, publicRuntimeConfig: { namespace: namespace, }, productionBrowserSourceMaps: true, poweredByHeader: false, assetPrefix: '', // we do not set this to our cache server on purpose, as it doesn't play nicely with plugin-static - we rather use proxy rules to redirect /_next/ to the cache server typescript: { ignoreDevErrors: true, ignoreBuildErrors: false, }, } module.exports = withPlugins(plugins, config) ```
• polyfills.ts

  ``` if (window.Promise && !window.Promise.finally) { window.Promise.finally = require('core-js/features/promise/finally') } require('intersection-observer') // next.js suggests to export undefined https://github.com/zeit/next.js/issues/7959 but this causes a babel warning. so we now export null and it seems to work export default null ```
• babel.config.js

  ``` module.exports = function (api) { api.cache(true) const presets = ['next/babel'] const plugins = [ [ 'module-resolver', { root: ['.'], alias: { '@my/alias': './some/path', }, }, ], [ 'styled-components', { ssr: true, }, ], '@babel/plugin-proposal-optional-chaining', '@babel/plugin-proposal-nullish-coalescing-operator', ] return { presets, plugins, } } ```
• plugin-static.js

  ``` * Thanks to Georges Haidar * * Registers 1) a module loader rule and 2) a handy alias for file-loader both * of which allow importing static assets in client-side code. * * 1) Module loader rule * This will allow importing images assets like any other module like so: * * ``` * import logo from "../static/logo.svg"; * ``` * * The value of logo will be a URL to the static asset * * 2) `static!` alias for file-loader * * The `static!` prefix can be used in import statements to treat a module as a * static asset and return a URL to it instead of bundling it and loading it * like a regular js/json module. * * ``` * import honeybadger from "honeybadger-js"; * ``` * The effect of this line would be that webpack bundles the node module and * the value of honeybadger will be the default export from it. * * What if instead we wanted a reference to this module as a url that can be * given to a ` Githubissues.
• Githubissues is a development platform for aggregating issues.