Using the source API, content written back into file is not rendered

[ndom91]

To Reproduce

So my goal is to take our releases.mdx page and on build, fetch the latest releases from the GH API and preppend the latest release markdown body to that releases.mdx page.

I very well may be doing it wrong, but here's what I've got. My source.ts:

export const { getPage, getPages, pageTree, files } = loader({
  baseUrl: "/",
  rootDir: "docs",
  source: createMDXSource(map),
  transformers: [{
    async ({ storage }) => {
      ...
    }
  }]
})

In the transformers[0] I'm doing:

1. Fetch GH releases content
2. Get releases.mdx page
   • const releasesFile = storage.files.get("releases.page")
3. Insert a new entry for the GH content into the releasesFile.data.data.exports.structuredData.contents array
4. Write it back
   • storage.write("releases.mdx", "page", releasesFile.data)

Current vs. Expected behavior

Current Behavior

The rendered /releases page only displays the markdown content in the releases.mdx file. None of the additional content from GH which I've written in via the sources API seems to appear

Expected Behavior

Additional GitHub content appears in the rendered page

Provide environment information

Operating System:
  Platform: linux
  Arch: x64
  Version: #1-NixOS SMP PREEMPT_DYNAMIC Thu May 30 07:45:04 UTC 2024
  Available memory (MB): 32026
  Available CPU cores: 12
Binaries:
  Node: 22.2.0
  npm: 10.7.0
  Yarn: N/A
  pnpm: 9.3.0
Relevant Packages:
  next: 14.2.4 // There is a newer version (14.2.5) available, upgrade recommended! 
  eslint-config-next: 14.2.4
  react: 18.3.1
  react-dom: 18.3.1
  typescript: 5.5.2
Next.js Config:
  output: N/A

Which area(s) are affected? (Select all that apply)

Core

Additional context

Using:

"fumadocs-core": "12.2.4",
"fumadocs-mdx": "8.2.33",
"fumadocs-ui": "12.2.4",

[ndom91]

Manually writing it back out (storage.files.set, like I found in the implementation for source.write) also doesn't seem to work :thinking:

  transformers: [
    async ({ storage }) => {
      // const releasesFile = storage.files.get("releases.page")
      const releasesFile = storage.read("releases", "page")

      if (!releasesFile) {
        console.log("No releases file found")
        return
      }

      // Fetch latest release from GitHub
      const ghResponse = await fetch(
        "https://api.github.com/repos/gitbutlerapp/gitbutler/releases/latest"
      )
      if (!ghResponse.ok) {
        console.log(`Failed to fetch latest release: ${ghResponse.statusText}`)
        return
      }

      const ghResponsePayload = await ghResponse.json()
      const latestRelease = Array.isArray(ghResponsePayload)
        ? ghResponsePayload[0]
        : ghResponsePayload

      if (latestRelease.draft || latestRelease.prerelease) {
        console.log("Latest release is a draft or prerelease, skipping update")
        return
      }

      // @ts-expect-error TODO type data
      const newContent = releasesFile.data.data.exports.structuredData.contents
      newContent.splice(1, 0, {
        heading: `v${latestRelease.name.replace("release/", "")}`,
        content: latestRelease.body
      })
      // @ts-expect-error TODO type data
      releasesFile.data.data.exports.structuredData.contents = newContent

↪️    // Both of these don't seem to work
      storage.write("releases.mdx", "page", releasesFile.data)
      storage.files.set("releases.page", releasesFile);
    }
  ],

I can only imagine that the structuredData object is not where new content should go, but I can't gleam where else it should be inserted

[fuma-nama]

Hmm are you trying to insert a entry to structured data? In this case you can modify the output instead of using a storage transformer. Also structured data is for implementing document search, you should use something else, like server components for auto-generated content

https://fumadocs.vercel.app/docs/mdx#built-in-properties

[ndom91]

Yeah maybe I'm using the wrong tool for the job. Like I mentioned above, my goal is to dynamically insert some content (a string of markdown formatted copy) into a pre-existing markdown file (/docs/content/release.mdx) during build.

What do you mean specifically by "modify the output instead of using a storage transformer"? Where would I do that?

[fuma-nama]

If you want to modify the structured data, add:

getPages().forEach(page => {
  page.exports.structuredData = ...
})

Instead.

[fuma-nama]

my goal is to dynamically insert some content (a string of markdown formatted copy) into a pre-existing markdown file (/docs/content/release.mdx) during build.

Use a Server Component to insert content dynamically