fails when trying to render around 100 routes

[pixelatorz]

when I try to render full website with around a 100 pages I receive: prerender-spa-plugin] Unable to prerender all routes!

first I thought It was some issue with some of the content, however when i manually copied the routes and commented out some it seems to work.

example: https://pastebin.com/Bg9Ke5cz

[tskimmett]

Try reducing the concurrency using maxConcurrentRoutes.

[pixelatorz]

yes that did indeed do the trick 👍

[carljustineoyales]

Try reducing the concurrency using maxConcurrentRoutes.

tried

maxConcurrentRoutes: 1,

still getting this error

[tskimmett]

It is possible your routes are failing for legitimate reasons (timing out, for example). However, if you are trying to render "a lot" of routes, it is also possible you're running into the same issues we ran into, specifically memory consumption and "random" socket hangups. We actually forked this repo and it's core dependency, prerenderer. We can successfully prerender +1000 routes using these forks, though beware that takes ~30 minutes for our application.

You're welcome to give the forks a try, with the caveat that we have no plans to support them beyond our own use. We may try to get our changes pulled into the main repos at some point, but it's not really a focus.

// package.json
"devDependencies: {
   ...
   "prerender-spa-plugin": "github:cognitoforms/prerender-spa-plugin",
   "@prerenderer/prerenderer": "github:cognitoforms/prerenderer"
   ...
}

// webpack config
config.plugins.push(new PrerenderSPAPlugin({
    staticDir: path.join(outputDir, 'assets'),
    outputDir,
    batchSize: 50,  // this is a new config property that controls how many routes are rendered before memory management code kicks in to avoid memory leaks with the prerenderer's internal express server
    routes: routes,
    renderer: new RendererPuppeteer({
        headless: true,
        renderAfterTime: 6000,
        renderAfterDocumentEvent: 'render-event',
        injectProperty: '__PRERENDER_INJECTED',
        inject: { renderInProcess: true },
        maxConcurrentRoutes: 10
    })
}))

This is unsolicited advice, but I would not recommend trying to deal with prerendering of more than a handful (<50) of routes. For such scenarios, you're probably going to be better off investing in getting server side rendering setup, as it is more scalable and seemingly less error prone, at least compared to these prerendering libraries.

[carljustineoyales]

when i first prerender 100+ routes for my project it works fine it fails about 2-3 times but retrying solves the issue, but after a week, i came to fix some layout issues and prerender it again, it fail, I retried multiple times but all failed

module.exports = (api, options) => {
    api.registerCommand("build:prerender", async (args) => {
        const { data } = await axios({
            method: "get",
            url: "<URL-API>",
            headers: {
                "X-API-KEY": "<X-API-KEY>",
            },
        });

        api.chainWebpack((config) => {
            config.plugin("prerender").use(PrerenderSPAPlugin, [
                {
                    // Required - The path to the webpack-outputted app to prerender.
                    staticDir: path.join(__dirname, "dist"),
                    // Required - Routes to render.
                    routes: ["/"].concat(
                        data.contents.map((post) => {
                            return `/post/${post.id}`;
                        })
                    ),
                    renderer: new Renderer({
                        injectProperty: "__PRERENDER_INJECTED",
                        inject: {
                            prerendered: true,
                        },
                        renderAfterElementExists: "[ready]",
                        headless: true,
                        ignoreHTTPSErrors: true,
                        maxConcurrentRoutes: 4,
                    }),
                },
            ]);
        });

        await api.service.run("build", args);
    });
};