[docs] Docs restructure

[siriwatknp]

From the decision in the Notion, the documentation will be restructured into each product.

Goal

Incrementally and smoothly transition to the new structure by preserving the same DX for the community and maintainers (tests should be added along the way).

The new repo structure:

Improvements are not the focus of this initiative. However, they can be listed in this issue and prioritized later.

Plan

To achieve a smooth transition, the work is split into 3 phases.

1. Preparation

Prepare E2E tests, migration tool, update scripts and add UI (each PR should be able to merge without affecting the production code, either set as feature flag or exclude from production build).

• [x] E2E tests for the website via playwright (playwright is already existed in the project). #30128
• [x] Create product pages config and update _app.js to switch between product. #30107
• [x] Migration script to duplicate pages to the new location (under /material/*). #30107
• [x] Update these scripts to compat with the new structure (it must still work with the current structure)
  • [x] docs:api:build #30245
  • [x] ~docs:i18n~ turns out this one needs no change because the translations directory remains the same 😊
  • [x] docs:typescript:formatted #30248
• [x] Update search to redirect to /material/* for material content. #30286
• [x] Refine scripts to generate pages with the decided structure (components & APIs) #30386
• [x] Create similar migration workflow for mui-x repo https://github.com/mui-org/material-ui-x/pull/3515
• [x] UI: Product identifier + versioning. #30283
• [x] UI: Product drawer. (Product drawer is open when clicking on the app icon) #30283
• [x] Deploy to a migration.mui.com to let ahrefs crawl to make sure that all of the links are working as expected. (need to turn enable_redirects on)

2.1 Migrate content

At this phase, the new URLs have to be deployed to production so that algolia can crawl and index. However, before algolia finish the indexing I think we should preserve the old URLs (meaning, people still browsing old URLs and won't realize that we are migrating the content).

• [x] Run the migration scripts for Core yarn docs:migrate:pages && yarn docs:api. https://github.com/mui-org/material-ui/pull/30757
• [x] Run the migration scripts for MUI-X yarn docs:migrate:pages && yarn docs:api. https://github.com/mui-org/material-ui-x/pull/3730

What'd happen after running the scripts

• data (demos, markdowns) will be moved from docs/src/pages/* to docs/data/*. (Both the old & new URLs are using the same demos, markdowns from the new location so we are always using the single source of truth)
• feature toggle enable_product_scope: true is turned on. This enables several things such as E2E tests for the new URLs, buildApi script, and query markdowns from the new location.
• product identifier & navigation are rendered on new locations but not on old URLs
• All of the links for existing pages remain the same, which means users won’t realize that the duplicated contents exist. (But they can accidentally access via direct URL).
• yarn docs:api will generate API pages for both previous & new URLs

3. New product spaces

Est. mid of Mar.

• [x] Rename material to /material-ui/* https://github.com/mui/material-ui/pull/31200
• [x] #31201
• [x] Set up a new Algolia crawler for indexing new URLs (old URLs are neglected). This new crawler has a different ID than the existing one. (old ID for existing URLs, new ID for new URLs, this way the redirect can be done at any time. https://github.com/mui/material-ui/pull/31178
• [x] Turn on feature toggle enable_mui_base and initialize MUI Base installation page. https://github.com/mui/material-ui/pull/30969
• [x] Refine the search experience for multiple product spaces https://github.com/mui/material-ui/issues/31253
• [x] Move MUI Base content from material to its own space https://github.com/mui/material-ui/issues/31414
• [x] Write a short announcement on the blog. https://github.com/mui/material-ui/pull/32044
• [x] Date pickers docs https://github.com/mui/mui-x/pull/3451
• [x] Remove date pickers from Material-UI https://github.com/mui/material-ui/pull/31810

4. Go live

Est. 4th April 2022.

• [x] Merge these PRs and deploy.
  • Core: https://github.com/mui/material-ui/pull/32048
  • X: https://github.com/mui/mui-x/pull/4313

5. Clean up

• [x] remove material-redirects.spec e2e after redirects period
• [x] delete old pages insidedocs/pages/...`
• [x] `remove material-old e2e test
• [x] remove replaceHtmlLinks, replaceUrlLinks
• [x] remove migration script
• [x] remove old config in buildApi.ts
• [x] use new logic in parseMarkdown.js for the API section

cc @mui-org/core @mui-org/x

[siriwatknp]

@danilo-leal @oliviertassinari @mbrookes Can you take a look at the images for product drawer UI (in the description above), does it make sense to you?

[mbrookes]

The button for opening product content is moved to the bottom-right of the page (similar to https://reactjs.org/docs/getting-started.html)

For ReactJS it seems that it's only a FAB at mobile breakpoint where a fixed Nav is no longer practical. I don't think having it placed bottom right permanently (if that's the intention) is good UX.

Even if positioned more conventionally, I'm not sure it will be immediately obvious with just the icon how to navigate between products. Perhaps the docs need a home page?

[siriwatknp]

Perhaps the docs need a home page?

How would that help?

Note: The mobile device is only ~3% from google analytics.

[mbrookes]

How would that help?

By having a top level page that allows for navigation between products, rather than depending on an obscure icon.

Note: The mobile device is only ~3% from google analytics.`

Then why would you chose to optimise for it by using a mobile-first design pattern on desktop? (bottom-right FAB as the primary navigation mechanism for moving between products)

[oliviertassinari]

The current plan looks great. A few more thoughts:

1. I think that we should move with as small steps as possible. As far as I understand it, we only need two docs to justify doing the restructuring, and the blog post. So we could do two, but anything extra (e.g. 4) would be even better. Now, my point is that I would challenge that delaying the release beyond Q1 2022 for having more docs has an overall positive ROI. Why? Because we wouldn't go to prod as fast as we can, and people tend to forget, and not notice things (we would be eventually in a great state with all the docs created).
2. I feel that https://mui.com/material-ui/react-modal/ would resonate better than https://mui.com/material/react-modal/. Am I the only one? My main concern is that https://mui.com/x/ works because the full product name is MUI X. But https://mui.com/material/, the product name is not MUI Material (decided to not have developers re-learn the existing brand name). I don't have a strong point of view on this but wanted to raise this potential opportunity in case it resonates with other team members. Related to https://github.com/mui/material-ui/pull/30969#discussion_r801464844

[mnajdova]

I feel that https://mui.com/material-ui/react-modal/ would resonate better than https://mui.com/material/react-modal/. Am I the only one?

+1 on this one

[siriwatknp]

I feel that https://mui.com/material-ui/react-modal/ would resonate better than https://mui.com/material/react-modal/. Am I the only one?

🥲 Another migration, but it looks like it should be mui.com/material-ui/*.

[siriwatknp]

@oliviertassinari Another thought, is it

https://mui.com/material-ui/react-modal/ or https://mui.com/materialui/react-modal/

? given that we decided to go with "Material UI" without a hyphen.

I had this thought after seeing https://headlessui.dev

[michaldudak]

Hyphens are commonly placed in URLs in place of spaces, so even though the name of the library is Material UI, to me it's perfectly fine to have material-ui in the URL.