search

zachleat / bench-framework-markdown

A set of scripts to test markdown processing speeds in various site generators/frameworks
34 stars 13 forks source link

Remix results #6

Closed kiliman closed 2 years ago

kiliman commented 2 years ago

It looks like your machine is not configured properly. Here's my test run on a MacBook Pro 16" 2019 2.4 GHz 8-core Intel Core i9 with 64GB RAM.

EDIT Removed results since the benchmark scripts were broken. It actually never copies the posts to the folder, since the folder is previously deleted, cp complains about missing folder. With the actual posts copied, the build times were definitely longer.

Like @jpcafe said on Twitter, this synthetic benchmark is not how Remix would handle markdown files. Those files should be treated as data and rendered on demand. Remix is not an SSG framework. Sure it supports MD routes, but mainly for convenience when you have a few markdown files. But if you have 4000, you'd keep those in a database, or in a lot of cases pull them from GitHub and render it on the fly. Either caching via Redis or using HTTP caching headers.

Anyway, I think it was disingenuous (considering you maintain one of the other frameworks) to release a benchmark result that was such an outlier without reaching out to the Remix community first.

maxcell commented 2 years ago

@kiliman He literally asked for them on Twitter, a site they frequent. https://twitter.com/zachleat/status/1553056554966040578

kiliman commented 2 years ago

Yeah, but posting the results before a response?

Anyway, turns out his benchmark script is broken. It removes the posts folder at the start, so the copy process fails with 'posts folder doesn't exist'

Anyway, fixing the script then re-running his benchmark does result in longer build times.

Like @jpcafe said on Twitter, this is not how Remix would handle MD files as it's not built for SSG site.

zachleat commented 2 years ago

We did have some good conversation with @jpcafe10 on Twitter! As I noted on there, the Remix benchmark does follow the conventions noted on their official docs: https://remix.run/docs/en/v1/guides/mdx

I think it’s worth cross posting our twitter conversation too! https://twitter.com/zachleat/status/1554488609008455681

zachleat commented 2 years ago

I pushed a small change to guarantee that the posts folder exists for fresh installs of the benchmark. This wouldn’t have affected the benchmarking results.

I think the actionable feedback has been resolved here, thanks!