search

squidfunk / mkdocs-material

Documentation that simply works
https://squidfunk.github.io/mkdocs-material/
MIT License
20.9k stars 3.54k forks source link

⚡️ New: Instant Previews #6704

Closed squidfunk closed 7 months ago

squidfunk commented 9 months ago

Instant Previews are finally here 😎 insiders-4.52.0 ships a new experimental feature that allows the user to preview another documentation page when hovering or focusing a link. You can test it on our documentation! Here's how it looks:

https://github.com/squidfunk/mkdocs-material/assets/932156/436bc067-955a-43f3-8c56-ea69fe2bb516

Sponsors can start using it right now! Why should I sponsor?

Instant previews can either be enabled globally, or for an individual link. Note that this feature is experimental. You can help us test it, so we can find the ideal mode of operation together, pushing it into a stable state. We've enabled it on selected links on our documentation in the reference guide, as well as on all version classifiers.

Usage

Per target page or section (recommended)

The master of Insiders contains a brand new Markdown extension that allows to scope instant previews to specific link targets, which can be pages or entire sections. This is explained in more detail in https://github.com/squidfunk/mkdocs-material/issues/6704#issuecomment-1959462904:

markdown_extensions:
  - attr_list
  - material.extensions.preview:
      targets:
        include:
          - changelog/index.md
          - customization.md
          - insiders/changelog/*
          - setup/extensions/*

Per link

Make sure to enable Attribute Lists, and add the data-preview attribute to any internal link:

[Link text](URL){ data-preview }

Globally

Add the following line to mkdocs.yml:

theme:
  features:
    - navigation.instant.preview

Note that this is not recommended, because it adds an instant preview to each and every link, which can be quite overwhelming. This option is essentially only provided for debugging instant previews themselves.

What's coming next

With your feedback, we will iterate on this feature, quickly pushing it into a stable state, before we will start working on the next big plugin that will come to Material for MkDocs: the glossary plugin. This plugin will allow you to create rich glossaries that go far beyond the rudimentary glossary functionality provided by Material for MkDocs, which will make your documentation more interactive and lift your documentation to the next level ⬆️

As always, we're happy for any feedback we receive from you, as it allows us to build the best possible version of this.

pawamoy commented 9 months ago

Awesome feature! I'm currently testing it, and I noticed a few occurrences of popup misplacement:

Enough space below Not enough space below
instant_preview_1 instant_preview_2

First arrow is where the link is, second arrow is where the window pops up.

I'll try to come up with a full repro in the next few days :slightly_smiling_face: Feel free to hide this comment!

squidfunk commented 9 months ago

Thanks for testing! If you could craft a minimal reproduction with steps how you reach that state, it would be amazing, because I cannot observe it. Happy to fix it asap!

shepherdjerred commented 9 months ago

It seems that you're using the sitemap to determine if an instant preview is available: https://github.com/squidfunk/mkdocs-material-insiders/blob/64b0e363e7e7c70c1f99a785d89f31f54c611f45/src/templates/assets/javascripts/components/content/link/index.ts#L149-L150

Is there any plan to remove reliance on the sitemap? We ship our docs as part of our on-prem software, so we can't know the site_url ahead of time.

Aside from that, it also looks like Material for MkDocs is now rewriting relative URLs to absolute URLs based on the page's origin. We had some CSS to style external links, but now that's now a bit more difficult with the URLs being converted.

But, this does look amazing! As always, thanks for adding such nice features :)

squidfunk commented 9 months ago

Excellent question. Yes, instant navigation, multi-language support, and versioning all need to rely on sitemap.xml, because we must know if a link points to another page inside the same (or related) documentation project. sitemap.xml is the only source of that information. Imagine if you host your documentation project under https://example.com/docs, and your API reference (possibly not built with Material for MkDocs) under https://example.com/reference. How should Material for MkDocs know that a link points to a site that can be replaced with instant navigation?

Thus, site_url is mandatory for this to work. The fact that it isn't is, IMHO, a historical oversight in MkDocs itself. If you cannot set site_url under any circumstance during the build, I recommend setting it to a value that you then replace with grep or some other text replacement tool when deploying. This is only necessary when you don't know the deployment at build time. IMHO, site_url should be required.

Aside from that, it also looks like Material for MkDocs is now rewriting relative URLs to absolute URLs based on the page's origin. We had some CSS to style external links, but now that's now a bit more difficult with the URLs being converted.

That is correct. We had to do that to fix bugs in instant navigation. I explained the necessary changes in detail in https://github.com/squidfunk/mkdocs-material/issues/2318#issuecomment-1909276860. Among other options discussed in this issue, the easiest and canonical way to do add an external icon to each link is to use the privacy' plugins external links feature (currently sponsors only):

plugins:
  - privacy:
      links_attr_map:
        rel: external

Then, add the following CSS:

.md-content [rel=external] {
  /* Your styles */
}

Sometimes, changes that break customizations are necessary to fix bugs that are otherwise not fixable. Instant navigation has so many edge cases, that we found this to be the only way forward. Of course, this sometimes upsets users, but you know how it is: change is hard. However, we need it to keep this project fresh and awesome.

Happy to receive more feedback on instant previews!

shepherdjerred commented 9 months ago

Thank you for the detailed response, Martin. I love breaking changes because that usually means great new features are being added :)

Is the sitemap.xml really the only way to know what documentation pages are being built? I'm not familiar with the MkDocs build process, but I'm very surprised there's not some way to retrieve a list of files being built.

To clarify, I do set site_url, but it points to our own hosted version of the docs. So, a customer might be hosting a copy of our products (and our docs) at customer-domain.com, and the sitemap points to official-docs.com.

I believe there was some limitation/reason why we weren't using that feature of the privacy plugin; I'll give it another shot.

squidfunk commented 9 months ago

Is the sitemap.xml really the only way to know what documentation pages are being built? I'm not familiar with the MkDocs build process, but I'm very surprised there's not some way to retrieve a list of files being built.

Instant navigation and instant previews are pure post-build in-browser features, so yes, it is the only source of truth. Only those pages can be accurately replaced/injected dynamically, because we can rely on the correct structure being present. You could, in theory, also inject external pages, but the results will not be consistent due to varying markup.

To clarify, I do set site_url, but it points to our own hosted version of the docs. So, a customer might be hosting a copy of our products (and our docs) at customer-domain.com, and the sitemap points to official-docs.com.

I'd say when you do this, you're in "undefined behavior" land. Note that site_url is used for much more than that, e.g., social cards will not work if your site_url does not match the actual URL on which it is deployed, and the canonical URL meta tags of your site will be wrong, which might lead to undefined behavior in the context of search engines. Thus, working around this issue will not solve all issues.

I believe there was some limitation/reason why we weren't using that feature of the privacy plugin; I'll give it another shot.

Happy to learn about that, and check if we can fix it.

squidfunk commented 9 months ago

One more addition:

To clarify, I do set site_url, but it points to our own hosted version of the docs. So, a customer might be hosting a copy of our products (and our docs) at customer-domain.com, and the sitemap points to official-docs.com.

The URL which is prefixing the entries in sitemap.xml is actually replaced upon resolution with the base URL the sitemap was fetched from. This is necessary, so deploy previews and serving content locally actually works. So, it might actually just work, but testing is required. The comment explains it for local usage:

https://github.com/squidfunk/mkdocs-material/blob/02bf848fd504b953acb1bdb8aad9531ac052c93a/src/templates/assets/javascripts/integrations/sitemap/index.ts#L49-L66

shepherdjerred commented 9 months ago

I'd say when you do this, you're in "undefined behavior" land. Note that site_url is used for much more than that, e.g., social cards will not work if your site_url does not match the actual URL on which it is deployed, and the canonical URL meta tags of your site will be wrong, which might lead to undefined behavior in the context of search engines. Thus, working around this issue will not solve all issues.

Cards actually do work for us!

Our a site that has a sitemap that points to docs.posit.co, but is hosted at p3m.dev/__docs__/admin. Of course, this only works because both sites have essentially the same content.

CleanShot 2024-02-02 at 17 39 27@2x

I understand that this is incredibly hacky, but to some extent it gives us desired behavior. We would rather search engines not list customer's deployments of our docs and instead point to our official docs site.

Happy to learn about that, and check if we can fix it.

I gave this a shot today and it worked great. I would love to see this capability added to the privacy documentation!

The URL which is prefixing the entries in sitemap.xml is actually replaced upon resolution with the base URL the sitemap was fetched from. This is necessary, so deploy previews and serving content locally actually works. So, it might actually just work, but testing is required. The comment explains it for local usage:

I confirmed today that this doesn't work. The anchor tags don't have the mouse enter/exit event handlers that appear on properly configured sites.

squidfunk commented 9 months ago

I gave this a shot today and it worked great. I would love to see this capability added to the privacy documentation!

I'm not sure what you mean. What should we add to the documentation of the privacy plugin / guide?

I confirmed today that this doesn't work. The anchor tags don't have the mouse enter/exit event handlers that appear on properly configured sites.

Could you provide a minimal reproduction with exact steps and expectations?

shepherdjerred commented 9 months ago

I'm not sure what you mean. What should we add to the documentation of the privacy plugin / guide?

Maybe it's too specific, but I would have found it useful if the privacy section mentioned that you can use it to apply CSS to external links. It's one of those things that I didn't realize CSS could do using the rel attribute.

Could you provide a minimal reproduction with exact steps and expectations?

To clarify, this feature works fine when the site_url is properly set. It does not work with my hacky setup where the site_url doesn't match the domain I'm hosting the docs at. Are you still wanting me to open an issue for this?

squidfunk commented 9 months ago

Are you still wanting me to open an issue for this?

You can attach a minimal reproduction to this issue so we can check if we can fix it, even though site_url does not match. In general, we don't consider this as an issue due to undefined behavior, but we can still check if we can make it work somehow, because in my head, it should work theoretically. Maybe it's only a small fix that we need to apply.

squidfunk commented 9 months ago

I've investigated and know why instant previews do not work when site_url is not set – it's essentialy a limitation of MkDocs, not Material for MkDocs. If you do not set the site_url, the page.canonical_url attribute is not set, which means the URL is never added to the sitemap. In fact, if you look at your sitemap, it is likely to be completely empty – that's at least my observation. Without the URL being part of the sitemap, we cannot be sure that a site is part of the current site, so we don't allow to replace it. Possible mitigations:

By the way, I'm pretty sure, instant previews should work for you when you use mkdocs serve, as this will populate the sitemap.xml with all pages prefixed with http://localhost:8000. However, if you build your project with mkdocs build and the sitemap contains URLs, and instant previews do not work, please provide a minimal reproduction.

shepherdjerred commented 9 months ago

Sorry, I haven't spent the time on making a reproduction yet, but I will note that the site_url is set.

e.g. this site: https://solo.rstudiopm.com/__docs__/admin/

has the site url set to docs.posit.co. The sitemap reflects this: https://solo.rstudiopm.com/__docs__/sitemap.xml

squidfunk commented 9 months ago

As said, a minimal reproduction would allow me to debug this, so if you find some time to compile one, that'd be awesome! The problem with tooltip positions being off in some cases was reported in #6735 by @pawamoy and is already fixed.

Let's find and squash those bugs quickly! ☺️

Guts commented 9 months ago

Really nice feature. Congrats and thanks again!

lampensau commented 8 months ago

I have been eager to test this out since the prototype was teased back in 2021 (?). I am glad to see it finally arrive 🎉. As always, thanks for your hard work!

Since I only have very little time at the moment, I just tried adding the data-preview attribute to a random internal link and found that it is not working for me. Since the project I've tried this on has many customizations, I will have to find time to take a deeper look. However, I thought it would be worth asking if there are any known incompatibilities with internal features or plugins (e.g., navigation.instant must be active for this to work).

I have, of course, checked the sitemap.xml and ensured the site_url is set correctly.

Additionally, I am wondering if there are plans to make this feature compatible with the projects plugin or external mkdocs-material instances/pages.

squidfunk commented 8 months ago

@lampensau are you sure you're running Insiders? If the link is internal, it should work out of the box without any extra configuration. You can try to enable navigation.instant.preview which will add it to all internal links and see if anything appears at all. Also check the network inspector. If that doesn't help, we're going to need a minimal reproduction, as the feature is still experimental and we might have missed some usage cases, because there are just so many.

However, I thought it would be worth asking if there are any known incompatibilities with internal features or plugins (e.g., navigation.instant must be active for this to work).

There are no known incompatibilities and no necessary changes to be made to mkdocs.yml or any other configuration. Also, enabling navigation.instant is not required.

Additionally, I am wondering if there are plans to make this feature compatible with the projects plugin or external mkdocs-material instances/pages.

We're open to exploring different directions. The projects plugin is definitely a prime candidate, and other external material-built sites are also something that could be added support for, and it shouldn't be too hard. We'd like to receive some more feedback however, before we go down further paths, so at first, instant previews are limited to the containing site.

remi-preghenella commented 8 months ago

Hi @squidfunk, Looking at your documentation, I've found that when you look at the instant preview and click on an anchor, the link will lead to a 404 page.

From https://squidfunk.github.io/mkdocs-material/setup/setting-up-navigation/?h=instant+previews#instant-previews, clicking on the "Using annotations" (see screenshot) will go to https://squidfunk.github.io/reference/annotations/#usage rather than https://squidfunk.github.io/mkdocs-material/reference/annotations/#usage

image

Probably a bug due to the fact that anchors are not processed by the instant preview.

Hope that helps!

squidfunk commented 8 months ago

@remi-preghenella Fixed in 663e148. Good catch! It seems I only have encountered relative links that stay on the same level when testing it, which would explain why it worked for me. URLs should now always be correctly resolved.

@pawamoy Additionally, bd6c45f lifts the restriction that instant previews did not work for links on the same page. This was reported to me on a private channel. Instant previews should now work quite nicely with mkdocstrings.

squidfunk commented 8 months ago

Both fixes were just released as part of 9.5.10+insiders-4.52.3.

Guts commented 8 months ago

Are Instants Preview supposed to work on tags located at the top the page?

I'm just asking because at the beginning, I found that a bit excessing since it's not really an internal link.

Then, after a few feedback with end-users about it, they found it pretty convenient.

Should it be an option?

squidfunk commented 8 months ago

@Guts

All internals links that are part of the main content are transformed into links if you enable navigation.instant.preview. Note that it is not recommended to do that, because, well, all internal links will be turned into instant previews, which can be quite overwhelming. A better idea is to deliberately attach data-preview, either manually or automatically to links that should be previewable.

In general I think it doesn't make sense to add instant previews everywhere, but it is quite impossible for us to decide where. Feature flags cannot take options, so this would need to be implemented outside of the features attribute. If you cobble together a hook or plugin, I'd be very interested what works for you. We'll be working on Markdown extensions and plugin that will leverage instant previews in the coming weeks/months, automatically adding data-preview where it makes sense.

On a more general note, please know that this feature is experimental. We're still in gravitation stage – learning what works for our users and what makes sense to add or remove. Happy for any feedback, good or bad. ☺️

Guts commented 8 months ago

@squidfunk Thanks, it's really valuable to have opportunity to discuss a feature publicly.

I get it's not recommended to enable it at website scale but I can't expect from writers to manage attribute lists on hyperlinks! I tried it hardly before (alignment on images, tooltip on hyperlinks...) and it's too complicated for someone who just come to share some technical stuff on our website. People has no "available brain time" to dive into every feature/syntaxe that Markdown and upper-layers introduce!

Some ideas maybe to make this feature even more customizable?

squidfunk commented 8 months ago

Please keep in mind that instant previews are not a plugin, but a frontend feature. It is specifically designed so that you can easily write a hook or plugin to add classes automatically where necessary. We'll be exploring whether we can provide a Markdown extension that can be configured to add those links automatically on specific elements shortly ☺️

Thanks, it's really valuable to have opportunity to discuss a feature publicly.

You can always open a discussion to discuss features! However, with the newest "flagship" features, we try to involve sponsors, so they can influence the direction it takes. Another great reason for sponsoring 😅

squidfunk commented 8 months ago

I've prototyped a new extension that I already pushed to Insiders in e406744, so we can test it together ☺️ This extension allows to define pages (sources) on which instant navigation should be activated, as well as links (targets) which should render instant previews. We've already enabled it on our documentation with the following settings:

markdown_extensions:
  - material.extensions.preview:
      targets:
        include:
          - changelog/index.md
          - customization.md
          - insiders/changelog/*
          - setup/extensions/*

The above configuration means: render instant previews for all links that point to the changelog, the customization page, the Insiders changelog, as well as all Markdown extensions. Render links from all pages, which is why sources is not defined.

We activated it on our documentation, so you can check it out already!

It is yet undocumented, but it follows the same logic as the inclusion/exclusion patterns for other plugins, like the social plugin. In general, only include should be necessary to define. You can use patterns to include whole subsections of your documentation, or only separate pages.

Again, this extension is highly experimental (I just prototyped it today), so use at your own risk. However, it should already be quite safe to use, since it only adds attributes when a pattern matches, and doesn't do any other alterations.

Happy for any feedback, good or bad!

Guts commented 8 months ago

Please keep in mind that instant previews are not a plugin, but a frontend feature.

You're right. I gave the glightbox example because I had it in mind but it could be obviously be a markdown extension.

Another great reason for sponsoring 😅

Just in case, I'd like to point out that I'm a sponsor (via an organization I admin and finance) despite the absence of a badge.

I've prototyped a new extension that I already pushed to Insiders in e406744, so we can test it together ☺️ This extension allows to define pages (sources) on which instant navigation should be activated, as well as links (targets) which should render instant previews. We've already enabled it on our documentation with the following settings:

Really nice! In my experience, it's really convenient to package an extension within a mkdocs plugin (as termynal does for example). And it would make it compatible with your group plugin.

I'll try to test your changes in next days.

squidfunk commented 8 months ago

Just in case, I'd like to point out that I'm a sponsor (via an organization I admin and finance) despite the absence of a badge.

That was meant as a general note that sponsors can influence the direction of features, exactly as you did in this conversation. We're very aware of your continued support and very thankful ❤️

In my experience, it's really convenient to package an extension within a mkdocs plugin (as termynal does for example). And it would make it compatible with your group plugin.

I deliberately decided to put this into an extension, as it's transforming the parsed AST, thus more lightweight than a plugin, and a plugin is not necessary at this point since we're only patching links. We'll be moving several parts of plugins to extensions in the future. I've only begun to write Markdown extensions, which is why up to now, most part is implemented as part of plugins. It's a journey for me as well.

squidfunk commented 8 months ago

The new Markdown extension has been released as part of 9.5.11+insiders-4.53.0. Additionally, I've updated our documentation how to configure automatic instant previews. 4.53.0 also brings pinning of blog posts 🤫

StarfallProjects commented 8 months ago

Hiya! Possible bug? It looks like at present you can't have both sources and targets?

For example, this works:

  - material.extensions.preview:
      sources:
        include:
          - release-notes.md

As does this:

  - material.extensions.preview:
      targets:
        include:
          - log-streaming.md

However this results in only the targets > include working:

  - material.extensions.preview:
      sources:
        include:
          - release-notes.md
      targets:
        include:
          - log-streaming.md

I've attached a reproduction:

9.5.13+insiders.4.53.1-instant-preview-issue.zip

If this isn't a bug, is there any chance of it being fixed anyway? It would be great to be able to set it up so all links in our release notes get previews, but for the site as a whole, we probably only want automatic previews pointing to one section.

squidfunk commented 8 months ago

Thanks for the reproduction. Yes, sounds like a possible bug from what you're describing. I'll look into it!

squidfunk commented 8 months ago

@StarfallProjects I've looked into it and everything is working as intended. Your configuration is:

markdown_extensions:
  - material.extensions.preview:
      sources:
        include:
          - index.md
      targets:
        include:
          - other-file.md

This means:

  1. Only analyze the pages in sources.include when adding instant previews. All other pages will not be touched and analyzed by the plugin, thus completely ignored. In this case, index.md is the only page that will contain instant previews, no other page will receive them.
  2. From the pages that remain after filtering with sources.include (and potentially sources.exclude) gathered in 1. (or from all pages, if the setting is omitted), analyze all links that point to a page included in targets.include and that are not excluded via targets.exclude (both only if present). Since only other-file.md is present in targets.include, it is the only file to which a link from index.md will render an instant preview.

On another note, all of those are set operations. If an include directive is omitted, it equals to ALL, whereas if exclude is omitted it equals to NONE, so all pages are included and no pages are excluded.

If you change the setting, the secondary file will also be rendered with a preview:

markdown_extensions:
  - material.extensions.preview:
      sources:
        include:
          - index.md
      targets:
        include:
          - other-file.md
          - secondary-file.md

The aim of the extension to give maximum control over where instant previews are rendered while making it as comfortable as possible at the same time. If you specify settings, those settings must be very explicit. We might add further filtering capabilities in the future if it makes, sense, but in general, the source and target is the most sensical one.

squidfunk commented 8 months ago

If this isn't a bug, is there any chance of it being fixed anyway? It would be great to be able to set it up so all links in our release notes get previews, but for the site as a whole, we probably only want automatic previews pointing to one section.

Re-reading your case, I think we could think about extending support to allow for multiple filter configurations, adding one more level of indirection. I haven't thought about such complex setups, but that's also why we're still considering this feature highly experimental. So, in essence, if I understand correctly, something like this (concept):

  - material.extensions.preview:
      configurations:

        # Preview all outgoing links in release notes
        - sources:
            include: [release-notes.md]

        # Preview all links on the entire site that point to log streaming
        - targets:
            include: [log-streaming.md]
StarfallProjects commented 8 months ago

@squidfunk thanks for clarifying. And yup, something like what you describe would be perfect.

squidfunk commented 8 months ago

@StarfallProjects implemented in 3d7136c. Could you please run the master of Insiders and check if this solves the problem for you? Ideally, give this a thorough test drive to learn whether it provides enough flexibility. If it proves to be a better approach, we'll go for it. Note that this is experimental so we might change it at any time, but we won't take functionality away obviously – just slightly gravitate on the syntax 😉 Thus, this should work now:

markdown_extensions:
  - material.extensions.preview:
      configurations:

        # Preview all outgoing links in release notes
        - sources:
            include: [release-notes.md]

        # Preview all links on the entire site that point to log streaming
        - targets:
            include: [log-streaming.md]
StarfallProjects commented 8 months ago

@squidfunk amazing, I'll test this Monday

StarfallProjects commented 8 months ago

@squidfunk realised my usual method of testing updates of course won't work as this isn't released yet. I had a quick look at the readme and contributing guide and can't see any guidance on building and testing the project? Do you have a link?

However, the behaviour you've described sounds exactly like what we're looking for.

squidfunk commented 8 months ago

I'm not sure what your usual method is, but you just need to install the master of Insiders:

pip install --force-reinstall git+https://${GH_TOKEN}@github.com/squidfunk/mkdocs-material-insiders.git@master

After testing, you can go back to any tag, e.g. the latest 9.5.13-insiders-4.53.1 like so:

pip install --force-reinstall git+https://${GH_TOKEN}@github.com/squidfunk/mkdocs-material-insiders.git@9.5.13-insiders-4.53.1

We haven't released or documented this yet, as I'm waiting for feedback, but once you confirm that this solves your problem, we're going to refactor the Markdown extension and make this the new default as it's more powerful.

StarfallProjects commented 8 months ago

Sorry for the confusion. I usually install it as a submodule, which I updated this morning. When it wasn't working I assumed it was picking up the latest release from the submodule rather than master . . . however I've now tried installing as normal using the GH token, and having the same issue. To be safe I created a fresh virtual environment, and the problem persisted.

My source is working - all links preview. But my target page doesn't instant preview when linked from another page.

9.5.13+insiders.4.53.1-testing-instant-preview.zip

Edit to add: to confirm I am indeed on master image

squidfunk commented 8 months ago

@StarfallProjects thanks for the reproduction. It turned out to be a one-line fix – I messed up continue with return in the last additions 😅 Fixed in 2024c82. As far as I understand, the behavior you're aiming for should work now:

Bildschirm­foto 2024-03-12 um 09 49 29
StarfallProjects commented 8 months ago

Awesome!

squidfunk commented 7 months ago

I'm archiving this announcement. If you experience any problems with instant previews, or have feedback to share with us, please create a new issue in case of bugs or discussion in case of questions and feedback ☺️