Nuxt 2.13.0 compatibility (nuxt static export)

[rutgerbakker95]

What problem does this feature solve?

It would be nice if the sitemap automatically generate routes in combination with the Nuxt static option

_{This feature request is available on Nuxt community (#c106)}

[mornir]

haha I was just going to request the same feature 😄 To be more precise, it would be nice if the sitemap module could work with the new crawler option in full static mode.

At first I didn't want to use the crawler because I assumed it would increase the build time. But actually there is no difference, at least in my case. So I removed my routes and let the crawler do the job, but then I realised that my sitemap was now empty. Of course, I could still query the routes for the sitemap, but it would be awesome if the sitemap module could simply take the routes crawled by the Nuxt crawler.

[lluishi93]

Yes please! This would be an awesome feature actually...

[mornir]

For fully static Nuxt websites hosted on Netlify, this build plugin could also be a viable solution: https://github.com/netlify-labs/netlify-plugin-sitemap

[lluishi93]

For fully static Nuxt websites hosted on Netlify, this build plugin could also be a viable solution: https://github.com/netlify-labs/netlify-plugin-sitemap

Woah! Didn't know about that one! Thank you!

[gabrielsze]

This will be super great!!!

[lomashbhattarai]

It would be great. Would save us from the pain of generating the routes manually

[lluishi93]

How is this feature going?

[willin]

  i18n: {
    locales: [
      { name: '简体中文', code: 'zh', iso: 'zh-CN', file: 'zh.js' },
      { name: 'English', code: 'en', iso: 'en-US', file: 'en.js' }
    ],
    strategy: 'prefix',
    rootRedirect: 'zh',
    defaultLocale: 'zh',
    lazy: true,
    langDir: 'i18n/',
    seo: true,
    detectBrowserLanguage: {
      useCookie: true,
      cookieKey: 'i18n_redirected'
    },
    vueI18n: {
      fallbackLocale: 'zh'
    }
  },
  sitemap: {
    hostname: 'https://hanzi.press',
    i18n: true
  }

sitemap generated:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:news="http://www.google.com/schemas/sitemap-news/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml" xmlns:mobile="http://www.google.com/schemas/sitemap-mobile/1.0" xmlns:image="http://www.google.com/schemas/sitemap-image/1.1" xmlns:video="http://www.google.com/schemas/sitemap-video/1.1">
  <url>
    <loc>https://hanzi.press/about</loc>
  </url>
  <url>
    <loc>https://hanzi.press/ar</loc>
  </url>
  <url>
    <loc>https://hanzi.press/card</loc>
  </url>
  <url>
    <loc>https://hanzi.press/story</loc>
  </url>
  <url>
    <loc>https://hanzi.press/surround</loc>
  </url>
  <url>
    <loc>https://hanzi.press/</loc>
  </url>
</urlset>

i have no idea whether it's about nuxt v2.13.0

[drewbaker]

Would love this to work with nuxt generate and the crawler

[colsen1991]

Second this! Although as a Netlify user, @mornir's comment about https://github.com/netlify-labs/netlify-plugin-sitemap seems to do the trick for now :)

[JohnFitz]

Until that's implemented, if you're not using Netlify and your requirements are modest, you could add this to your nuxt.config.js:

//nuxt.config.js

import fs from 'fs'
import path from 'path'

export default {
    hooks: {
        generate: {
            done: (generator) => {

                const hostname = 'https://pinkyandthebrain.com' // Your hostname

                // Array of regular expressions to exclude routes. If you don't need to exclude anything, just use an empty array
                const exclusionRegExps = ['\/admin\/.+', '\/secrets.*', 'planstotakeovertheworld'] 

                const sitemapFilePath = path.join(
                    generator.nuxt.options.generate.dir,
                    'sitemap.xml' // Customise sitemap name here
                )

                const filteredRoutes = Array.from(generator.generatedRoutes).filter((route) => {
                    return exclusionRegExps.some(rule => RegExp(rule).test(route)) ? false : true
                })

                const urlElements = filteredRoutes.map(route => {
                    return `<url><loc>${hostname}${route}</loc></url>`
                })
                const template = `<?xml version="1.0" encoding="UTF-8"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd">${urlElements.join('')}</urlset>`
                fs.writeFileSync(sitemapFilePath, template)
            }
        }
    }
}

Caveats

1. This is all of the code, i.e. this doesn't use the sitemap module at all. Because it's so simple, you don't get all of the nice features like gzipping, sitemap indexes, globbing etc., it just spits out a simple sitemap including your dynamic routes, minus whatever routes match the regular expressions in the exclusionRegExps array.
2. Logic in the config smells bad, so I'm sure this isn't considered idiomatic Nuxt, but I'm new to Nuxt and I needed the feature fast so I had to cut corners. When dynamic routes can be added using the sitemap module, it'd probably be better to use it over this solution.
3. This uses regex to exclude routes so as with anything regex, you have to look out for corner cases. Check the output to make sure.

Pros

It doesn't get any simpler than this, so it should be easy for anyone to customise :wink:

[mornir]

@timbenniks also wrote a blog post about it: https://timbenniks.dev/writings/easy-dynamic-routes-in-your-nuxt-sitemap/ I guess it shouldn't too hard now to have that code directly in this module.

[timbenniks]

@mornir thanks for the mention. It should indeed not be too complex to add my approach to the module. However, I don't know about its internals and potentially there is quite a bit to consider. Like multiple sitemaps etc.

[drewbaker]

@mornir Thanks for that! I just implemented it, but I realized you can use the default sitemap option filter to exclude routes, so you don't need to define them in the build module like you have it. Just keep all config in the nuxt.config file this way.

[drewbaker]

I tired making a PR for this, but kept getting Git commit issues. Never done a PR to an OSS project before.

Oh well I give up. So frustrating what has become of frontend DX these days.

Pretty sure this is the solution if someone wants to make a PR. Will need to add generatedRoutes to /lib/options.js too. And tests and documentation.

/lib/module.js:

const path = require('path')

const fs = require('fs-extra')

const { generateSitemaps } = require('./generator')
const logger = require('./logger')
const { registerSitemaps } = require('./middleware')
const { getStaticRoutes } = require('./routes')

module.exports = async function module(moduleOptions) {
  const nuxtInstance = this

  // Init options
  const options = await initOptions(nuxtInstance, moduleOptions)
  if (options === false) {
    logger.info('Sitemap disabled')
    return
  }

  // Init cache
  // a file "sitemap-routes.json" is written to "dist" dir on "build" mode
  const jsonStaticRoutesPath = !nuxtInstance.options.dev
    ? path.resolve(nuxtInstance.options.buildDir, path.join('dist', 'sitemap-routes.json'))
    : null
  const staticRoutes = fs.readJsonSync(jsonStaticRoutesPath, { throws: false })
  const globalCache = { staticRoutes }

  // Init static routes
  nuxtInstance.extendRoutes((routes) => {
    // Create a cache for static routes
    globalCache.staticRoutes = getStaticRoutes(routes)

    // On run cmd "build"
    if (!nuxtInstance.options.dev) {
      // Save static routes
      fs.outputJsonSync(jsonStaticRoutesPath, globalCache.staticRoutes)
    }
  })

  // On "generate" mode, generate static files for each sitemap or sitemapindex
  nuxtInstance.nuxt.hook('generate:done', async (context) => {
    await nuxtInstance.nuxt.callHook('sitemap:generate:before', nuxtInstance, options)
    logger.info('Generating sitemaps')
    await Promise.all(options.map((options) => generateSitemaps(options, globalCache, nuxtInstance)))

    // Add Nuxt's auto generated routes
    options.map((opts) => {
      const output = {
        ...opts,
      }
      if (opts.generatedRoutes) {
        const routes = Array.from(context.generatedRoutes)
        output.routes = [...routes]
      }
    })

    await nuxtInstance.nuxt.callHook('sitemap:generate:done', nuxtInstance)
  })

  // On "ssr" mode, register middlewares for each sitemap or sitemapindex
  options.forEach((options) => {
    registerSitemaps(options, globalCache, nuxtInstance)
  })
}

async function initOptions(nuxtInstance, moduleOptions) {
  if (nuxtInstance.options.sitemap === false || moduleOptions === false) {
    return false
  }

  let options = nuxtInstance.options.sitemap || moduleOptions

  if (typeof options === 'function') {
    options = await options.call(nuxtInstance)
  }

  if (options === false) {
    return false
  }

  return Array.isArray(options) ? options : [options]
}

module.exports.meta = require('../package.json')

[NicoPennec]

Thank you all for your suggestions! I know this feature is eagerly awaited.

I just published a PR to handle the new static mode directly in the sitemap module. You can test this proposal by installing it manually from the github branch, as follows:

$ npm install nuxt-community/sitemap-module#feat/static-crawler

Your test results and feedbacks would be really appreciated before to release it? 🙏

[mornir]

@NicoPennec Thanks for implementing this feature! It works great. Is it better to add this module to buildModules instead of modules, so that this module code doesn't end up in the production bundle? I've added it to buildModules and there was no error.

[rutgerbakker95]

@NicoPennec very nice! In my case it also works great.

[Velikolay]

@NicoPennec that's really cool, worked well on my side too.

[Tragio]

@NicoPennec thank you for the amazing work. What is missing to be merged? 😃

[Velikolay]

@NicoPennec does the feature respect the exclude option? I couldn't make exclude work on routes generated with the crawler.

[LukaHarambasic]

@Velikolay as there is a new test added without a crawler and the old one still tests with an exclude I would say, yes it is expected to work.

https://github.com/nuxt-community/sitemap-module/pull/170/files#diff-f91577e8fb8dcd2f38cbffaf99bd21cae2be72d4ee60e0bbcecb11de6d15d171R743

[willin]

not compatible to @nuxt/content-theme-docs

nuxt config:

import theme from '@nuxt/content-theme-docs';

export default theme({
  loading: { color: '#00CD81' },
  i18n: {
    locales: () => [
      {
        code: 'zh',
        iso: 'zh-CN',
        file: 'zh-CN.js',
        name: '简体中文'
      }
    ],
    defaultLocale: 'zh'
  },
  buildModules: ['@nuxtjs/google-analytics', '@nuxtjs/google-adsense', '@nuxtjs/sitemap'],
  content: {
    liveEdit: false
  },
  components: true,
  pwa: {
    manifest: {
      name: 'RxJS教程'
    }
  },
  googleAnalytics: {
    id: 'UA-33096931-4'
  },
  'google-adsense': {
    id: 'ca-pub-5059418763237956',
    pageLevelAds: true
  },
  sitemap: {
    hostname: 'https://rx.js.cool'
  }
});

result: https://rx.js.cool/sitemap.xml

<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:news="http://www.google.com/schemas/sitemap-news/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml" xmlns:mobile="http://www.google.com/schemas/sitemap-mobile/1.0" xmlns:image="http://www.google.com/schemas/sitemap-image/1.1" xmlns:video="http://www.google.com/schemas/sitemap-video/1.1">
<url>
<loc>https://rx.js.cool/</loc>
</url>
<url>
<loc>https://rx.js.cool/releases</loc>
</url>
</urlset>

source: https://github.com/willin/rx.js.cool

[urbgimtam]

Hoping this PR gets released soon. Its a must for larger static websites.

[fabianwohlfart]

+1 Any News?

ps. Feature works great on nuxt 2.15.4 with content, i18n, 6 level deep urls, 200+ pages.

[patrickcate]

Is it better to add this module to buildModules instead of modules, so that this module code doesn't end up in the production bundle?

I'd like to know this as well.

[honeyamin]

is is for static or work by ssr? also which version of nuxt? and also can we get dynamic routes from apollo?? thanks every body that who read this and answer. also thanks to all

[pixelomo]

I gave up trying to get the crawler feature working, wasted 1-2 hours with this module then found a chrome extension that could generate a sitemap for SPAs with zero configuration - Sitemap Generator

When it finishes crawling save the sitemap.xml and move it your /static folder. When you run npm run build it will be added to your /dist folder

[marvinhuebner]

Thank you all for your suggestions! I know this feature is eagerly awaited.

I just published a PR to handle the new static mode directly in the sitemap module. You can test this proposal by installing it manually from the github branch, as follows:

$ npm install nuxt-community/sitemap-module#feat/static-crawler

Your test results and feedbacks would be really appreciated before to release it? 🙏

Is there any update on this and when will this be merged? Would be great to have that.