nuxt / content

The file-based CMS for your Nuxt application, powered by Markdown and Vue components.
https://content.nuxt.com
MIT License
3.08k stars 624 forks source link

10000 content files and dynamic slug page results in very long page load in SSG scenario (ssr:false, clientDB:true, nuxt generate) + huge cache file #2587

Closed kevinhaeni closed 6 months ago

kevinhaeni commented 6 months ago

Environment

Darwin Node v18.16.0 CLI 3.8.3 Nuxt/Content: 2.12.0 Nuxt: 3.10.3

Reproduction

We have the following structure in the content folder:

[..slug.vue]:

<script setup>
const { data: pageContent, error } = await useAsyncData(`content-${route.path}`, () => getPageContent()));

async function getPageContent() {
   const doc = await queryContent().where({ _path: route.path }).findOne();
   return doc;
}
</script>

We also have a index.vue which does something similar with homepages under /content/homepages/

Here is the nuxt.config:

import { globby } from 'globby';
import svgLoader from 'vite-svg-loader';
const configHelper = require('./utils/configHelper/configHelper');

const languages = configHelper.getLanguages();
const { locales, defaultLocale } = configHelper.getLocales(languages);

// since the nuxt3 crawler doesn't seem to properly work, we need to provide a list of routes
const getRoutes = async () => {
  const LANGUAGES = ['de', 'fr', 'en', 'it', 'rm'];
  const CONTENT_DIR = '../collector/content';

  // generate a list of routes for all pages in the content directory/<lang>/<slug>
  const filePaths = await globby(`${CONTENT_DIR}/[a-z][a-z]/*.json`);
  const routes = filePaths.map((path) => {
    const parts = path.slice(CONTENT_DIR.length + 1).split('/');
    const [lang, slug] = parts;
    return `/${lang}/${slug.slice(0, -5)}`; // Remove '.json'
  });

  return [...routes];
};

// https://nuxt.com/docs/api/configuration/nuxt-config
export default defineNuxtConfig({
  nitro: {
    prerender: {
      crawlLinks: true
    },
  },
  // @nuxtjs/i18n module configuration
  i18n: {
    // Detect browser language & previous language by cookie
    detectBrowserLanguage: {
      useCookie: true,
      cookieKey: 'i18n_redirected',
      redirectOn: 'root',
    },
    strategy: 'prefix',
    langDir: 'languages/',
    locales,
    defaultLocale,
    vueI18n: './vueI18n.js',
  },
  // Nuxt modules
  modules: [
    '@nuxt/content',
    '@nuxtjs/i18n',
    '@pinia/nuxt',
    'nuxt-prepare',
    '@nuxt/test-utils/module',
  ],
  // @nuxt/content module configuration
  content: {
    sources: {
      content: {
        driver: 'fs',
        // Path to rendered content
        base: '../collector/content',
      },
    },
    experimental: {
      clientDB: true,
    },
  },
  ssr: false,
  hooks: {
    async 'nitro:config'(nitroConfig) {
      if (nitroConfig.dev) {
        return;
      }
      const slugs = await getRoutes();
      //@ts-ignore
      nitroConfig.prerender.routes.push(...slugs);
      return;
    },
  },
});

Describe the bug

In the generated dist and on every page load, there is a 17.7MB cache.1749238406456.json file loaded that contains the whole content folder (around 10'000 JSON Files) and causes a massive delay in the pageload.

Is that intended or are we doing something wrong?

Additional context

No response

Logs

No response