Build time grows with every render

[codeserk]

Hello, I'm using this lib to render MD/MDX from remote sources and it works quite well. However, I noticed that the build time grows with every render. I have a page that uses the Markdown component a few time and it goes crazy in dev mode:

The first render is fast-ish 100ms but the time keeps growing with every refresh of the page 300ms - 600ms - 1200ms at some point it's just too slow and the only solution is to restart the dev server. This behavior also happens when the app is built and I start the server (npm run start).

I'm opening the issue in case this rings a bell and/or you know the possible cause somehow, I'll try to debug it better and come up with a PR if I find the cause

Thanks !

[Adammatthiesen]

How are you fetching your data? I will note that we use a version of this in StudioCMS with the AstroDB integration for storing the Mardown strings in the Studio Database and that works pretty well and doesn't have to much delay,

I will also note, that i don't have multiple Markdown components on the same page, so i'm not sure exactly on that note, but it sounds like something you could remedy if you could merge these entries together in a way that would make sense(if its possible to only parse and render from the <Markdown> component once. that should help a bit with the speed)

[codeserk]

In my use-case I have md for the summary in my posts:

---
title: Post title
summary:
  type: md
  raw: Summary with _some_ md **stuff**
---

The normal _md_ content (rendered by astro)

I'm using astro to render the posts (since they are actually not remote at all), but I need to use something else to parse the md from summary so I decided to use this lib to test if it would work for my use-case when I have fully remote md

^ this is an example of my use-case, I have some posts (for languages in this case), but in the index I want to get all and render summary for each.

The issue is not about how much it takes to render (at least the first time), the problem is that it's slower with every page refresh.

15:22:10 watching for file changes...
15:22:13 [200] / 149ms
15:22:14 [200] /_image 67ms
15:22:14 [200] /_image 69ms
15:22:15 [200] /_image 34ms
15:22:15 [200] /_image 50ms
15:23:36 [200] / 104ms
15:23:37 [200] / 195ms
15:23:38 [200] / 430ms
15:23:39 [200] / 888ms
15:23:41 [200] / 1392ms
15:23:44 [200] / 2056ms
15:23:48 [200] / 2974ms
15:23:54 [200] / 4187ms
15:24:01 [200] / 5452ms
15:24:12 [200] / 7313ms
15:24:24 [200] / 8874ms
15:24:42 [200] / 12109ms

and at some point I have to restart the dev server to make it usable -> this means it will only work for SSG and not SSR, since the same problem happens after build

My feeling is that the component is doing something that is permanently adding complexity to the parse, like adding a new vite plugin or something every time (?) - really have no idea how it works :(

I'll debug a bit more and I'll create a way to reproduce, hopefully I find what it is :)

[Adammatthiesen]

Its just using Marked.js to parse the markdown(this can be used both Client-side and Server-side), there is not really anything fancy for this one... (no vite plugins or anything that should be doing adding any delay...)

Yeah That is a weird delay that i've never seen in all of my projects that are using astro-remote, or any derivative... In fact i actually use astro-remote in SSR more than in SSG. Makes me wonder what it triggering the delay.

[codeserk]

@Adammatthiesen I think I found the culprit, although I'm not sure what is the best solution:

https://github.com/natemoo-re/astro-remote/blob/main/packages/astro-remote/lib/utils.ts#L112

marked plugins seems to be applying every time the Markdown component is rendered, so every time the page is refreshed, more plugins are stacked into the render phase, making it slower I have an extreme use-case for this (51 Markdown renders in the home :D) - but I think the same problem can happen even with just 1 per page, only slower

However, I'm not sure what should be the solution for this, since the plugin and components can vary from one Markdown render to another 🤔

[codeserk]

@Adammatthiesen After reading a bit about marked I think I might have found a solution for this case 🤔 The utils is currently using marked global object from the library, but a new instance can be instantiated also.

Instead of:

  marked.use(markedSmartypants(), markedFootnote(), ...markedExtenstion, {
    gfm: true,
    renderer,
  })
  const content = await instance.parse(dedent(input))

it could do this:

  const instance = new Marked(markedSmartypants(), markedFootnote(), ...markedExtenstion, {
    gfm: true,
    renderer,
  })
  const content = await instance.parse(dedent(input))

what do you think? if you think is fine I can create a PR

UPDATE: forgot to mention, this change fixes the problem I was facing

[Adammatthiesen]

@natemoo-re This could be a good chance to add an Optional Integration with a Global Config object (and a Markdown component for this config) to use for this type of configuration.

@codeserk I'm not sure if changing the main Component utility would be the best plan, as astro-remote currently does not act like an integration, its a simple component library, where you could create strange side-effects if one component had a extension but another didn't. (This is one reason StudioCMS uses a global config object instead of Unique entries as well)

[codeserk]

@Adammatthiesen aren't side-effects happening with the current utils ? Since it uses the global marked, whatever plugin is used outside of the utils will be also used for parsing, right?. That's why I was suggesting using instance instead of global marked 🤔

[codeserk]

@Adammatthiesen I've created a PR with the proposal solution for the problem I mentioned, I created a test to, where the problem can be reproduced (removing the patch in utils.ts)

[m4rrc0]

Hi. Early report of a huge build performance hit on a project of mine. I tracked the issue down to version 0.3 of this package. Version 0.2.4 builds my 700 pages in ~50s on my machine, while v3 keeps going forever. I have got a lot of pages building off of the same routing file, first few page build times are 5 to 20ms but the build time keeps growing and growing.

I tried applying @codeserk 's PR #35 and it fixes my issue.

Let me know if I can test something on my side to help.