search

squidfunk / mkdocs-material

Documentation that simply works
https://squidfunk.github.io/mkdocs-material/
MIT License
20.03k stars 3.46k forks source link

Material 5.0.0 Status #1306

Closed squidfunk closed 4 years ago

squidfunk commented 4 years ago

TL;DR This issue is meant for updates and discussions on the progress of the rework of the theme. Please feel free to checkout and try the refactor/rxjs-typescript branch and comment on the current state of development.

Description

The last major releases of Material for MkDocs were mainly issued for compatibility reasons. While version 2 and 3 were related to backward incompatible changes of MkDocs (0.17.1 and 1.0 respectively), version 4 was released due to backward incompatible changes related to users with Chinese system languages (#911). The next iteration, version 5, is a substantial rewrite of the underlying JavaScript functionality to a TypeScript and RxJS-based architecture. This will yield the following major benefits:

  1. All features that are provided by the theme (e.g. the sidebars, tabs, search, etc.) will be observable by 3rd party JavaScript. This allows for very easy extension and customization. This will address issues like #1102 where users want to extend the functionality of the theme without re-compiling or forking it. Also, disabling functionality should be easier.

  2. Improving search performance. Search will be moved to a web worker and exhibit better caching and re-building behavior. This will mitigate problems were the UI freezes due to large search indexes which pre-building on the server-side only partly solves. Furthermore, search will be re-architected to fit into the plugin concept introduced by MkDocs 1.x.

  3. Color customization will be re-implemented using CSS variables. Changing the color to your custom brand colors will be even simpler and the theme will become smaller (as the palette CSS file will not be necessary anymore). Support for CSS variables is pretty great by now, but there are some browsers that don't support it, respectively IE 11, Opera mini, QQ browser and Baidu browser. These browsers will receive a (probably neutral) fallback color. If you must support those browsers, you can still extend the theme and customize the build will custom CSS colors as already described in the customization guide.

There might also be some slight face lifting, but the main aspects of Material 5 is the introduction of a more modern architecture. The rewrite will also make the theme testable, so unit tests can and will be written to ensure functionality.

I will document my progress here and there will be a beta phase similar to before the release of 1.0.0 (#46). Some lists with things that need to be done will follow.

Evaluate

TODOs

5.0

General

Functionality

General
Scroll-related
Sidebars
Search
Accessibility
Repository

Issues

5.x

wilhelmer commented 4 years ago
squidfunk commented 4 years ago

@wilhelmer nice idea 😁 I actually already thought about it. However, listening to the latest React Podcast with Kitze I learned that having a Donate button will probably not help you fund the work of your project. People don’t tend to donate for something that they can already use for free which is why I decided not to include one.

wilhelmer commented 4 years ago

Well you‘d get at least 1 pizza, better than nothing 😄

facelessuser commented 4 years ago

1 🍕 > 0 🍕

Yup, checks out.

squidfunk commented 4 years ago

I started to document my progress regarding the rewrite of the current features in the original post.

Furthermore I think I finally got the grasp of RxJS and boy, is it beautiful. If somebody has some experience with RxJS, I would really appreciate some comments on the architecture I'm building on the refactor/rxjs-typescript branch. See this folder.

After migrating everything to RxJS, implementing instant loading (aka "turbolinks" which I prototyped three years (!) ago here) will be a breeze, as re-binding components will be super easy. This would also greatly improve search UX, as the page will more or less behave like an SPA, and thus the search index has to be built only once.

squidfunk commented 4 years ago

I took the time and implemented instant loading as can be seen in the latest version of refactor/rxjs-typescript. All internal links that are not anchors (i.e. #...) are intercepted and dispatched via XHR. Then, the new components are extracted from the parsed response, replaced in the DOM and all downstream observables/listeners are re-bound.

Works pretty well already and it's super fast! v5 will add instant loading as an experimental feature similar to tabs. Some open questions remain, e.g. what to do with Google Analytics, Disqus and custom JavaScript but it should all be solvable.

wilhelmer commented 4 years ago

I had to look up what instant loading is, but now that I know: Sounds great! 👏🏻

squidfunk commented 4 years ago

@wilhelmer don’t really know what to call it. There’s this JavaScript library called Turbolinks which does something similar but that does sound very Wordpressy.

Stanzilla commented 4 years ago

Do you want tickets for v5 bugs yet?

I only found two small problems so far:

squidfunk commented 4 years ago

@Stanzilla thanks for checking it out already. For now just post your findings here. If we hit beta / release candidate, we can open a separate issue and maybe project to keep track of things without flooding the issue tracker.

https://github.com/squidfunk/mkdocs-material/blob/refactor/rxjs-typescript/src/base.html#L422 this will cause a console error, needs src="{{ 'assets/javascripts/app.js' | url }}"

What do you mean exactly? It's dependent on the app.js script, that's correct. Same behavior as before, except everything exported by the script is exported into the global variable app. I think of probably exporting all watch* functions into the global namespace but haven't decided yet.

I think https://github.com/squidfunk/mkdocs-material/blob/refactor/rxjs-typescript/src/assets/javascripts/index.ts#L92 should be as string[] and then > https://github.com/squidfunk/mkdocs-material/blob/refactor/rxjs-typescript/src/assets/javascripts/index.ts#L132 names as [] just started learning TS though.

Thanks for catching them. Typings are ongoing :-) The component union type must be put somewhere else, will fix the typings then. Maybe don't focus on the code base (e.g. typing errors) right now but on the functionality because I sometimes commit stuff that is not quite ready yet / needs work, but the branch is WIP anyway.

How does the instant loading work for you? I think it works pretty great already, though a little jittery when skimming through the browser history. Still trying to map out how to smoothen things out.

Stanzilla commented 4 years ago

Alright! And I was getting some "app is not defined" errors before but I just checked again and they are gone, sorry!

Functionality is pretty nice atm, I haven't checked which of my workarounds from v4 I can drop, I made some to workaround the collapse issue for examples. Also I edited the partial so I can have external links open in a new window instead, that currently causes a few JS errors in the console. Is that something you would like to support natively in the future? Maybe as a parameter from mkdocs.yml?

squidfunk commented 4 years ago

Functionality is pretty nice atm, I haven't checked which of my workarounds from v4 I can drop, I made some to workaround the collapse issue for examples.

The collapse issue (will) be fixed completely as we'll now listen on the change event and not click like in v4.

Also I edited the partial so I can have external links open in a new window instead, that currently causes a few JS errors in the console.

Good to know, will look into it. CMD + Click should not be intercepted but open in a new tab as expected.

Is that something you would like to support natively in the future? Maybe as a parameter from mkdocs.yml?

Nope, we removed it for UX reasons. It's however super easy to add the behavior in v5 via custom JavaScript that runs automatically, probably something like:

merge(load$, switch$)
  .subscribe(document => {
    document.querySelectorAll("a").forEach(el => {
      el.setAttribute("target", "_blank")
    })
  })

... whereas load$ (initial render) and switch$ (subsequent render) are exposed by the application.

Stanzilla commented 4 years ago

That's slightly less ugly than my 

{% if nav_item.title == "A" or nav_item.title == "B" %}
    <a
      href="{{ nav_item.url | url }}"
      title="{{ nav_item.title | striptags }}"
      class="md-nav__link" target="_blank"
    >
    {% else %}
    <a
      href="{{ nav_item.url | url }}"
      title="{{ nav_item.title | striptags }}"
      class="md-nav__link"
    >
    {% endif %}
      {{ nav_item.title }}
    </a>

:D

Oh and I had to add the webpack copy plugin back for a custom JS file.

squidfunk commented 4 years ago

I moved everything that is unnecessary out of Webpack into the Makefile for much faster subsequent builds, as the Makefile correctly tracks dependencies and does not recompile everything upon a single change. Also copying, removing and replacing stuff is now entirely handled by the good ol' friends of UNIX.

Stanzilla commented 4 years ago

Just thought about the external links thing again, would still love to see that come back as an option because I'd still have to flag certain links with manual edits in the template for your JS solution to work properly.

squidfunk commented 4 years ago

The only external links that are added directly through Material are those in the footer and the source link. The Navigation is entirely local, so it should open in the same window. Furthermore, opening something in a new window is bad UX, as it's not transparent to the user. For this reason I can definitely say it's not coming back, I'm sorry.

Stanzilla commented 4 years ago

Yeah I just want to open social links and the edit one in a new window

markallasread commented 4 years ago 
* [ ]  Provide a link to donate to @squidfunk so he can order pizza and have more time for coding

Not-so-much that there'll be more time for coding, just that we can compensate you for your contributions. I don't expect more by donating, I already received everything I need! :)

squidfunk commented 4 years ago

@wilhelmer @markallasread I started an Amazon wish list – in case somebody wants to give something back. 😊

outofphase commented 4 years ago

@squidfunk the wish list link takes me to amazon.com...are you in the US?

squidfunk commented 4 years ago

@outofphase nope, Germany. You're correct. I'll fix it and migrate it to Amazon Germany.

squidfunk commented 4 years ago

Migrated the lis to my German account and fixed the link above, can you check again?

outofphase commented 4 years ago

Yes that's worked, although not surprisingly everything is now in German which made navigating through the Amazon maze a little tricky for me. I really hope that I haven't just started a 30 day free trial of Amazon Prime. With luck you should get the book by the XKCD guy on Weds.

I think that one of the Amazon prompts was saying "buying from somewhere else?" which I should have clicked, but that also was in German so I wasn't sure. You should probably give guidance about that for non-German speakers.

squidfunk commented 4 years ago

@outofphase thanks a lot, that's super nice! 😊

Changing the wish list locale doesn't seem to be possible according to a StackExchange post which I find to be very strange. When I created the wish list on the .com domain, all shippings would have been from America which would mean astronomical shipping cost, so I think in the end this is the better option. As I have a German Amazon account I'm not prompted with the question you received, so don't really know what guidance to give 😐If somebody encounters this, a PR to the README would be super awesome, where I also added this link.

coliff commented 4 years ago

You could preset the locale to English with the URL: https://www.amazon.de/hz/wishlist/ls/U7LAO2O62KBU?language=en_US&sort=default That'd make things a bit easier for non-German/English speakers. There is a change language option at the top of all Amazon sites though (to the right of the sidebar), so I think most people shouldn't have a problem...

squidfunk commented 4 years ago

@coliff thanks! Interestingly, en_US drops be back into German, but en seems to work.

wilhelmer commented 4 years ago

@coliff thanks! Interestingly, en_US drops be back into German, but en seems to work.

This is probably because technically, all five major Amazon EU sites (de, gb, es, it, fr) are on the same platform, while Amazon US is a totally different thing. So en is attributed to the GB site, while en_US is unknown to Amazon EU.

squidfunk commented 4 years ago

@wilhelmer thanks for the explanation. I just hope that en will cater to most cases.

wilhelmer commented 4 years ago

Enjoy your Masterminds of Programming ... und frohe Weihnachten :-)

squidfunk commented 4 years ago

@wilhelmer wow, thanks a lot! I will post a pic on Twitter within the next days showing what I received from you guys 😊meanwhile I'm working on v5, hope to finish an RC this year.

wilhelmer commented 4 years ago

You're welcome! It's a used copy, but in "excellent" condition. Let me know if there are any issues with that. I really don't want to cheap out, but I had very good experiences with these so-called "used" books in mint condition.

squidfunk commented 4 years ago

Some years ago, I bought a used copy of The Tao of Programming from 1987 which was (and still is) in perfect shape, so no remorse!

squidfunk commented 4 years ago

Just rolled out a 5.0.0-preview. It would be awesome to collect some feedback here, so if you have some time over the holiday, give it a spin! 😊This release is focused on the new search.

Note that I'm on vacation from Dec 25 to Jan 9.

wilhelmer commented 4 years ago

Omg christmas came early! Thanks! Super excited. Won’t have enough time during the holidays, but will definitely provide feedback before your return on Jan 9.

Stanzilla commented 4 years ago

random observation:

squidfunk commented 4 years ago

@Stanzilla was thinking about removing it anyway, as it may introduce indeterminism due to the re-ordering it does.

Stanzilla commented 4 years ago

Okay and what would you say is the best way of introducing custom files that need to be copied from node_modules to the material dir now, without reintroducing copy-webpack-plugin. Just manually copy them in the makefile?

squidfunk commented 4 years ago

@Stanzilla my recommendation would be to do it exactly like in the new Makefile.

If you want to copy files one-by-one:

https://github.com/squidfunk/mkdocs-material/blob/c736cd9939e1e208b4836cc8e305cafb14173ecf/Makefile#L73-L78

If you want to copy a whole directory:

https://github.com/squidfunk/mkdocs-material/blob/c736cd9939e1e208b4836cc8e305cafb14173ecf/Makefile#L89-L93

squidfunk commented 4 years ago

After replacing FontAwesome with SVGs, I'm also thinking about ditching the Material Icon font for the same. Could be quite tricky with the Admonition styles though, as they would need to be inlined in the CSS and SASS doesn't provide any filesystem helpers, but could be worth it.

Any thoughts?

squidfunk commented 4 years ago

9d67cdfe adds support for custom repository icons – it can now be set to any FontAwesome icon. The new default is brands/git-alt:

config:
  extra:
    repo_icon: brands/gitkraken
Bildschirmfoto 2019-12-23 um 20 32 39
squidfunk commented 4 years ago

Just released 5.0.0-preview.2. Merry Christmas!

Stanzilla commented 4 years ago

Only just now noticed that they are killing TSLint https://github.com/palantir/tslint/issues/4534

squidfunk commented 4 years ago

@Stanzilla I'm aware of TSLint being deprecated. I also tried to switch to ESLint for v5, but it seemed that the support is not quite there yet. I may try this again in the future, but for now it is TSLint. As linting is (mostly) a matter of style, we can introduce ESLint after v5 without breaking anything.

Stanzilla commented 4 years ago

Currently the compilation fails if you don't prefix social links with "brand/", might be something to fix to ease migration.

  <a href="{{ social.link }}" target="_blank" rel="noopener" title="{{ social.type }}" class="md-tabs__link md-tabs__social">{% include "assets/images/icons/fontawesome/" ~ social.type ~ ".svg" %}</a>
      File "/usr/lib/python3/dist-packages/jinja2/loaders.py", line 187, in get_source
        raise TemplateNotFound(template)
    jinja2.exceptions.TemplateNotFound: assets/images/icons/fontawesome/discord.svg
squidfunk commented 4 years ago

It's gonna be a breaking change which will be part of the upgrade guide lines. I don't think we should default to brands.

wilhelmer commented 4 years ago

As discussed in https://github.com/mkdocs/mkdocs/pull/1805, Material v5 breaks support for mkdocs-localsearch.

To support the plugin and enable local search (file:// protocol), I most probably need changes on your side.

The following code works without any changes on mkdocs-localsearch (hooray), but is probably super bad since I have no experience with RxJS at all.

You'll have to change the ajax() call in index.ts to:

const data$ = new Observable<AjaxResponse>()

if (window.shim_searchIndex) {
  data$ = ajax({
    url: ``
  })
    .pipe(
      map(response => {
        response.response = window.shim_searchIndex
        return response
      })
    )
      .pipe<SearchIndexOptions>(
        pluck("response")
      )
}  
else {
  data$ = ajax({
    url: `${config.base}/search/search_index.json`,
    responseType: "json",
    withCredentials: true
  })
    .pipe<SearchIndexOptions>(
      pluck("response")
    )
}

If mkdocs-localsearch is not installed or enabled, the theme works as before. What do you think?

squidfunk commented 4 years ago

@wilhelmer would it work, if we provide a configuration option that is a function returning a promise resolving with the search index? You would only need to change the initialization of the theme:

app = initialize({
  ...
  search: {
    index: () => Promise.resolve(window.shim_searchIndex)
  }
  ...
})

Internally, the theme will use the promise returned by the function instead of the XHR call.

wilhelmer commented 4 years ago

Sounds good, but that needs changes on index.ts too, right? If you provide the full code, I'd be happy to test it.

wilhelmer commented 4 years ago

Only tested my solution in FF – sadly, Chrome causes more problems :-(

Chrome doesn't allow web workers locally. For more info and a solution, see https://stackoverflow.com/questions/21408510/chrome-cant-load-web-worker

In our solution (see this PR), we used the worker-loader package with the option inline set to true to get around that. I think that option has the same effect as the solution posted on Stack Overflow.

squidfunk commented 4 years ago

Sounds good, but that needs changes on index.ts too, right? If you provide the full code, I'd be happy to test it.

@wilhelmer yes, this was only the general idea. We could also define a new block called config in the base.html template which allows for easier overriding of the configuration.

In our solution (see this PR), we used the worker-loader package with the option inline set to true to get around that. I think that option has the same effect as the solution posted on Stack Overflow.

Inlining web workers would blow up the main application bundle and lead to a longer FMP. I'm also not using the worker-loader because it led to problems with the TypeScript typings and is not necessary. The current solution uses multiple Webpack entrypoints with the same result. If you need inlining, you could fork the theme and override the Webpack configuration.

I'm sorry but we cannot optimize the theme for entirely local execution. It's an edge case which affects probably less than 1% of users. However, I'm happy to provide hooks and mechanisms to help you achieve a better solution than monkey-patching fetch. Providing a new template block to change the configuration and a new option for the configuration to load the index from a different source (i.e. the promise) should help you achieve exactly what you want.