search

fastify / website-metalsmith

This project is used to build the website for fastify web framework and publish it online.
https://www.fastify.io
MIT License
76 stars 133 forks source link

Investigate Docusaurus (revisited) #291

Open Xhale1 opened 2 years ago

Xhale1 commented 2 years ago

Prerequisites

🚀 Feature Proposal

Related: #100 and https://github.com/fastify/fastify/issues/722.

I propose revisiting migrating the docs to Docusaurus, especially after significant improvements to their tech since previous discussions.

I would love to draft up a proof of concept for the team if there's serious interest, and help with efforts to migrate if the demo proves viable.

Motivation

My team recently spent some time investigating alternatives to Express and some elements of the Fastify docs gave us pause. The most pressing for us was the lack of order and headers in the sidebar which made getting started more cumbersome than necessary. Additionally, something about the "system defaults" approach to Fastify's docs makes the site feel less professional than, say, Create React App or React Redux.

I think Docusaurus's default styling, creature comforts, and additional features make migrating a compelling option. Recent (since #100) support for markdown will make migrating far less troublesome than it once was.

Potential benefits:

Example

No response

Eomm commented 2 years ago

In general, I think the fastify website tech stack is out of date and need an upgrade.

I think the "must have" features are:

Nice to have:

I'm not against any new framework (Docusaurus, Gatsby, GitBook or others), I prefer the one that fits the fastify needs.

EDIT: forgot to say: we need someone that would like to push this initiative. I'm happy to give feedback, but I don't have a time slot to help implement this

mcollina commented 2 years ago

I recognize the problems in the documentation and current website. We truly need somebody that will champion it in the long run.

This is a significant endeavor. I'm not convinced in moving to Docusaurus is the right solution as it felt too "strict" in the past with how they wanted docs.

Xhale1 commented 2 years ago

My company is adopting Fastify and I would love to do what I can to lead new documentation improvements (content, design, infra, etc).

I suppose my next question is what do you feel would be the best next steps? I'm confident in Docusaurus as the best tool for the job though I could draft up some comparisons if it'd be beneficial.

I can build out a proof of concept for new docs as a jumping off / talking point for further conversations on what this would entail.

Edit: made it a bit more clear that I'm interested in leading documentation improvements.

jsumners commented 2 years ago

With the recent addition of support for guides in the documentation I intend to do a reorganization. I just haven't had time to get started yet. Might have some time this weekend. Not sure.

As for the proposed package: I would rather stay far away from Facebook tech.

Xhale1 commented 2 years ago

Regarding the Facebook tool: I'm generally all for anti-facebook sentiments but I think we should pursue the best tool for the job. Avoiding Facebook also means avoiding Next, Gatsby, and React.

That said I have some ideas for alternatives:

It seems to me that next.js and vue could end up requiring too much developer time, which would leave GitBook and Docusaurus as popular feature-rich options. I prefer the open-source nature of Docusaurus to GitBook, and I like the Next.js solution if the engineering time ends up being low enough.

jsumners commented 2 years ago

Avoiding Facebook also means avoiding Next, Gatsby, and React.

This is completely acceptable to me.

I'm not convinced we need to change the tooling. I will try to put together my proposal in short order. It amounts to:

  1. Split /docs into /docs/guides and /docs/reference
  2. Add /docs/index.md
  3. Add /docs/reference/index.md
  4. Move appropriate docs into the correct locations

A subsequent PR(s) would address the actual text of the documents. All reference docs should match the style of https://www.fastify.io/docs/latest/Encapsulation/ and https://www.fastify.io/docs/latest/Hooks/ .

lachlancollins commented 2 years ago

Hi, I've been directed here from fastify/fastify#2877. The key focus of my comment there was improving the integration of TypeScript documentation so that it is alongside and "equal" to the JavaScript documentation, rather than in its own separate page.

One of my suggestions as part of this improvement was implementing toggles for code examples between JS and TS, citing Redux Toolkit's docs as an example, which I've since learnt actually uses Docusaurus. I know @Xhale1 has mentioned improved code blocks as one of the benefits, and I see this toggle functionality as the next-step on top of that and another pro for migration.

In terms of the React ecosystem, at this point it simply has the best tools for frontend development, and I believe migrating to Docusaurus would ensure the tech supporting Fastify's documentation is future-proofed. I've also just checked and found that Metalsmith hasn't pushed any updates since 2016.

As someone with a fair bit of time on my hands at the moment, I'd be keen to work on a Docusaurus project for Fastify - let me know if this is something you'd be interested in.

Xhale1 commented 2 years ago

I feel strongly that avoiding react because it't facebook tech is a poor decision. The same logic could be applied to avoiding nodejs, typescript, github, npm, yarn, or a great many leading tools that make developer lives so much better.

A massive amount of the top tech sites in the world use React, and a healthy majority of top open source projects use Docusaurus. The project is fully open source and we could fork it to avoid facebook influence if necessary.

Adopting an industry standard tool makes first-time contributions far easier, and can ultimately let us focus on writing incredible documentation instead of re-inventing the technology itself. The benefits we would get out of the box seem extremely compelling to me, and they seem compelling enough to a large number of other open source projects.

I greatly appreciate the work that has gone into the existing documentation site and the Fastify ecosystem as a whole. That said, I believe docusaurus could benefit the community in a number of ways and is the best tool for the job.

Examples of incredible tools using docusaurus:

Edit: I want to readdress and refocus on the core pain points I think we're looking to address here.

Documentation features

Improving the in-house implementation will inevitably lead to a game of catch-up with tools dedicated to docs, and it will pull developer focus away from other parts of the Fastify ecosystem. Existing dedicated solutions give us new features to write amazing docs out-of-the-box, for free. We only pay the cost of migration.

Time commitment

Dedicated tools bring with them their own knowledge base and documentation on how to use them. Adopting a dedicated tool means our community benefits from the existing knowledge on how to use it. Docusaurus's own docs, for example, lower the barrier to entry to working on any docusaurus site and alleviates the struggle of learning the quirks of an home brewed system. In my opinion community members wanting to contribute to improving the docs site would have an easier time with docusaurus or similar than they would with the current system.

mcollina commented 2 years ago

There is a fundamental issue with docusaurus: versioning. Versioning in Docusaurus is very far from how things should be structured. Docusaurus support this as an afterthought by keeping all the data on the same repository (ref https://docusaurus.io/docs/versioning). The approach that this website currently has is great for this as it allows multiple versions to coexists (they are assembled from different releases).

IMHO we should just move this to elevently. It's more up to date than the current stack but it's still true to the same principles.

Note that the majority of the work is needed on the documentation itself.

jsumners commented 2 years ago

Note that the majority of the work is needed on the documentation itself.

This is correct. Unfortunately, I was just too out of it over the weekend to work on https://github.com/fastify/website/issues/291#issuecomment-971901732 (I started a local branch and that's as far as I got). Thursday and Friday are a holiday for me, so I hope to get some work done at that time.

In terms of the React ecosystem, at this point it simply has the best tools for frontend development, and I believe migrating to Docusaurus would ensure the tech supporting Fastify's documentation is future-proofed.

This statement is a matter of opinion. If you ask the folks that like Vue, Marko, or other things, they would say the same sort of thing about those technologies.

I feel strongly that avoiding react because it't facebook tech is a poor decision. The same logic could be applied to avoiding nodejs, typescript, github, npm, yarn, or a great many leading tools that make developer lives so much better.

It is my opinion that using Facebook tech is tacit approval of that company's practices and tactics on a myriad of issues around privacy and data collection. I am but one vote amongst many, but my vote will always be against using Facebook tech.

Adopting an industry standard tool makes first-time contributions far easier, and can ultimately let us focus on writing incredible documentation instead of re-inventing the technology itself.

On this we agree. I just want to see a different technology used to fill that role. Matteo is suggesting https://www.11ty.dev. I use https://gohugo.io for my personal sites and find it fantastic.

Remember, fastify.io is more than just documentation. It is a full website that needs to support things like https://github.com/fastify/website/pull/194 in addition to the documentation. The requirements set forth by https://github.com/fastify/website/issues/291#issuecomment-971494581 are what should be focused on in any project that seeks to update the website generation. For example, when I do get my proposal PR up for the documentation, I will have no way of knowing if it works since we do not have a way to generate test sites from PRs to the main repository.

lachlancollins commented 2 years ago

Just a few thoughts on the versioning issue in Docusaurus - wouldn't it be possible to just use the current strategy of clone all releases, and then copy the docs of old versions into /versioned_docs, and the latest release into /docs?

Also to clarify my view, I would like to see something more than the current site's features, including those mentioned by @Xhale1 up the top of this page. Docusaurus seems to be the easiest way to get these out-of-the-box and with less manual work and also has wide adoption. However, if you guys would be more comfortable with something like 11ty or Hugo, I'm not fussed as long as they can be expanded to add this functionality.

I'm also just wondering about the benefits of having separate doc versions for each minor release - most sites I've seen just split it into major. Is there a reason why this format is used on Fastify's website? I'm updating on v3 whenever I notice a new version, and haven't had any breaking changes (yet).

Thanks in advance - this is the first time I've been invested enough in an open source project to contribute to discussion, and I'm really looking forward to seeing how Fastify grows :)

jsumners commented 2 years ago

Please see https://github.com/fastify/fastify/pull/3474 for my proposal as a starting point.

jsumners commented 2 years ago

I'm also just wondering about the benefits of having separate doc versions for each minor release - most sites I've seen just split it into major. Is there a reason why this format is used on Fastify's website? I'm updating on v3 whenever I notice a new version, and haven't had any breaking changes (yet).

Personally, I would be willing to cull all of the old documentation and stick with the current supported LTS releases. Either at the semver major level or only the most recent release in the LTS line. It would greatly simplify a lot of site generation logic.

Xhale1 commented 2 years ago

I fully agree with @lachlancollins on not being attached to docusaurus, whatever tool gets us the best docs would be great. Apologies if I seem like I'm just pushing a single tool.

Regarding versioning: I also agree that we should drop the patch (and I think minor) versions of the docs. I propose having 1.x, 2.x, 3.x, and next (or similar). This is also the way docusaurus is built to handle versioning, with a next branch and then a set of docs for each major version. I actually really like how they handle it, but to each their own.

I also want to throw Vue Press into the mix for consideration as an alternative tool. I don't have very much experience with it but it's used extensively in the vue ecosystem.

jsumners commented 2 years ago

I think the best path forward is to start new repository, say new-website, that starts from scratch (but imports all necessary resources) as a proof-of-concept. This repo could then be promoted to the main repo upon completion.

This new project must:

lachlancollins commented 2 years ago

@jsumners I found the cross-referencing you mentioned in Hugo very cool and agree it is essential - I checked and found that Docusaurus also checks for broken links and fails with errors during build.

In terms of the framework of choice for new-website, I'm a bit uncertain on the verdict at this point - is Docusaurus entirely ruled out as an option because it is part of the React ecosystem?

EDIT: Another thought has come to me, to use Hugo's cross-referencing would we need to use its syntax in the main repo's /docs folder? I know @Eomm mentioned that this folder should remain the primary source earlier.

Xhale1 commented 2 years ago

This new project must:

I'm not sure this fully covers the grievances listed here.

It seems to me that the consensus is that the current Fastify docs tech stack is out of date and would require too much dev work when existing solutions are so readily available and prevalent in the open source community.

Some of the headlining issues appear to be

Additional issues related to the docs in general:

Please let me know if I'm missing something or if you believe one of these issues to be irrelevant.

That said, I again want too point out that it seems to me that Docusaurus would solve every issue here while also bringing additional improvements to the table, all without the needed developer lift of engineering new in-house solutions. The primary issues listed here with docusaurus are its versioning system and that it's owned by facebook. It seems that the versioning problem wouldn't be an issue for us and I'm not sold that avoiding tech due to its parent company is reason enough to avoid a solution if it provides enough value.

Eomm commented 2 years ago

Regarding versioning: I also agree that we should drop the patch (and I think minor) versions of the docs. I propose having 1.x, 2.x, 3.x, and next (or similar)

Some users opened bugs for not working features, but the issue was a wrong minor version. I think a good approach would be tracking only major versions, but having a UI that shows something like added in 2.34.0. That would be cool!

EDIT: Another thought has come to me, to use Hugo's cross-referencing would we need to use its syntax in the main repo's /docs folder? I know @Eomm mentioned that this folder should remain the primary source earlier.

I mean: we should use that documents' content, but it is not denied to update them to be processed by the new-site (for example adding markdown metadata).

Please let me know if I'm missing something or if you believe one of these issues to be irrelevant.

Your list is good

I'm not sold that avoiding tech due to its parent company is reason enough to avoid a solution if it provides enough value.

I'm a dumb programmer: right now, I read only PROs for Docusaurus 😄 Many questionable companies pay fantastic people to craft great tools that we are still using today since humanity invented wars. I think it is a matter of "do we use that tool to do good or evil?"

mcollina commented 2 years ago

Screenshot 2021-11-23 at 12 58 36

I think the versioning support in Docusarus is so bad that even them are not using it: if you flip that switch at the top it redirects to a different netlify URL. I can only see Docusaurus as problematic long term.

A better approach would be to have:

  1. a "product" website that includes:
    • guides
    • ecosystem
    • tutorials
    • companies using it
    • good first issues
  2. a rendered version of the reference documentation from fastify/fastify for each individual version

This will keep the build time short and free us from using a single technology for everything.

lachlancollins commented 2 years ago

The versioning on docusaurus's site is weird - they only directly build the current (beta 9) and previous beta (beta 8) in the live site. I imagine this is to keep their bundle size small while they iterate through so many small beta releases. However if you look at sites such as https://socket.io/ they don't have such problems, and the solution is very clean in my opinion.

If their specific versioned docs strategy doesn't work for our use case, we don't actually need to use it - it's totally possible for us to just split it into docs/latest/, docs/2.x/, etc. within the one folder, like how the current site handles it.

Would you be willing for either myself or @Xhale1 to at least build a quick demo to try to prove versioning is a non-issue and that docusaurus could be used across the whole website's infrastructure?

mcollina commented 2 years ago

I have invited both of you to a repository "website-next".

luisorbaiceta commented 2 years ago

Having gone through the struggle of working with the current tech stack when implementing the nested documentation, I totally agree with @Eomm. As a contributor I ended up spending a lot time trying to understand the current build process, and I think this can become a huge barrier when seeking help and contributions.

IMO, Jamstack is the go to solution, and as suggested by @mcollina Eleventy would be a good approach. Moving the current site would be less painful than adopting solutions like Next.js, Gatsby (or any other React framework), and having to atomize the whole website. Besides, Eleventy already supports the templating language we are already using (Nunjucks)

As for the linking problem described by @jsumners, I haven't had the time to get hands on this subject, but I already found some documentation in the eleventy page.

lachlancollins commented 2 years ago

Hi @luisorbaiceta, I agree that something like Eleventy is a much closer match to the current site's Metalsmith and would have very few cons - however, from what I can tell it wouldn't actually add much value to the user-facing side without additional work. In terms of Docusaurus, it brings docs enhancements that either work-of-the-box or with minimal configuration, including those mentioned by @Xhale1.

Also, Docusaurus falls within the Jamstack family :) As someone with a lot of experience with Next.js I agree that it would be more work compared to Eleventy, but have no doubt that won't be the case for Docusaurus.

luisorbaiceta commented 2 years ago

@lachlancollins I haven't had the chance to look at docusaurus, but for what I've read in this thread It has some issues with version handling. Is that correct? Is there any workaround?

lachlancollins commented 2 years ago

@luisorbaiceta yep we have discussed the versioning, and I explained why I'm not worried about it here.

To summarise, https://socket.io/ is a great example of using Docusaurus's versioning system, but there is no obligation for us to use that system, and we can keep organising it into subfolders of /docs/ based on release as the current site does.

Xhale1 commented 2 years ago

Hey @lachlancollins, I kinda jumped the gun and added a first pass of the docusaurus demo over at https://github.com/fastify/website-next. Worked on it while traveling and I didn't have wifi, hence no communication.

It's just the getting-started page for now which ended up being a good demo page. We're able to show off the use of tabs for NPM / Yarn and ESM / CommonJS, code block file naming (sooo nice imo), a long page with a table of contents, and basic formatting.

I think a good next step is getting raw opinions and doing a quick demo of versioning. Thoughts?

localhost_3000_docs_

lachlancollins commented 2 years ago

Hi @Xhale1, I just realised we have access to the main branch - I've been working on the docusaurus-demo branch and have done quite a bit :) just trying to merge our work now!

Xhale1 commented 2 years ago

@lachlancollins oh no, I'm so sorry, I'll check out your work now. If you want feel free to scrap my stuff (I only put an hour or so into it).

lachlancollins commented 2 years ago

@Xhale1 all good, it seems like you prioritised the config and I prioritised the docs so it worked out well :) just checking that the merge had no problems right now - if you notice anything missing please add it back in!

lachlancollins commented 2 years ago

Everything is building successfully, the big things which haven't been customised are the non-docs pages and the order of docs. Everything else seems pretty small. Should we make an issue/discussion over on https://github.com/fastify/website-next for progress instead of continuing on here? I might write a GitHub Action to let people browse the demo, @mcollina is this okay? If so can you handle the settings changes to enable GitHub Pages?

image

mcollina commented 2 years ago

I have enabled gh-pages. Please open issues on that repo if you'd like to gather feedback about that, this issue is already way too long.

lachlancollins commented 2 years ago

@mcollina final question, what is the URL for gh-pages? https://fastify.github.io/website-next/ doesn't work because it isn't at the root path.

mcollina commented 2 years ago

@lachlancollins please open a separate issue on that repo, describe Exactly what you need and let's keep this for high-level coordination/discussion. I'm going to hide this and a few of the previuous posts.

delvedor commented 2 years ago

I'm fine with docusaurs, as long as we can have some custom pages like we are having right now. A plus of docusaurs is that it's a well know tech, and maintaining it shouldn't be that hard. Still, we don't have experience with it, so if we want to commit to moving to docusaurs, I would kindly ask @Xhale1 @lachlancollins @luisorbaiceta if you are willing to help us maintain this for the foreseeable future. If not, then move to eleventy and figure out how to fix the link it's the best strategy IMHO.

luisorbaiceta commented 2 years ago

I can't really commit to a tech I am not familiar with either. I will migrate and help with the scripts and docs processing

lachlancollins commented 2 years ago

Hi @delvedor, I'm in a position right now where I'd be very happy to commit to maintaining the new website with Docusaurus! Also, I'd be more than happy to work alongside @Xhale1 if he's interested, and of course glad to have @luisorbaiceta helping with the scripting :)

Before we continue working on this, is there anyone else who needs to give this change the thumbs up? Thanks!

Xhale1 commented 2 years ago

I can commit to maintaining the docs site as well, super happy to work with @lachlancollins. The nice thing is most of the tech is largely maintained by the greater docusaurus community letting us focus on the docs themselves (which I'm also really interested in improving).

webketje commented 1 year ago

Hi there,

I found this repo through GH's "Used By" section on a metalsmith plugin. There are numerous elaborate efforts linked to this repo to switch to vitepress, fastify+vite or docusaurus, which obviously provide more out-of-the-box features than Metalsmith. As the late maintainer of the latter (since sept. 2021), I can of course only encourage you to continue using metalsmith and support its development through PR's where you find features lacking or new ecosystem plugins. FWIW I also co-maintain the underlying jstransformer dependencies used by some ms plugins.

For a "quick win update" though (= eliminating security issues, better editor intellisense, ESM/CJS builds, TS support and extra, backwards-compatible features) you could switch metalsmith-layouts to @metalsmith/layouts, and metalsmith-sass to @metalsmith/sass. New plugins have been added too (@metalsmith/js-bundle an esbuild js bundler, @metalsmith/table-of-contents, @metalsmith/postcss).

Most issues listed in https://github.com/fastify/website/issues/291#issuecomment-976212036 can be fairly easily implemented with Metalsmith plugins/ templating too. I have been experimenting with docs in metalsmith for https://metalsmith.io/docs. For inspiration the source code is at https://github.com/metalsmith/metalsmith.io/. A.o. there are:

Whichever route you choose to go, thank you for having chosen metalsmith in the past and up until now :smile: