Reorganising and improving plugin documentation

[seankmartin]

📚 New content request

Hello everyone!

The napari plugin ecosystem and the content at napari.org/plugins have been growing rapidly, and it would be beneficial to reorganize the plugin documentation to improve navigation. Here is a proposal for restructuring the existing plugin-related content and introducing new valuable information. This layout has already been shared with @DragaDoncila and the initial round of feedback has been incorporated into this proposal. Please add any suggestions or comments regarding this proposal - I'd love to hear your thoughts!

Mock-up of the plugins main page

Here is a mock-up of the proposed site layout at a top level:

Proposed plugin documentation structure

Click to view the full plugin site layout proposal


---

Click to view the proposed table of contents

* Getting started * Finding and installing plugins * What plugins can provide * How to use plugins * Getting support for plugins * Tutorials * Filtering plugins on napari hub * Using plugins for cell segmentation * Converting a napari script into a plugin * Building a full plugin * Building a plugin * Your first plugin * Plugin anatomy * Plugin contributions * Key concepts for plugins (e.g. multithreading) * Debugging during plugin development * Best practices * User experience guidelines * Overview of user experience * Consistency with napari * Menus and tooltips * Containers and tabs * Testing and publishing * Testing guidelines * Describing and categorising your plugin * Setting up the napari hub plugin preview page * Enhance your plugin's reach and demonstrate quality * In depth-guide to plugin testing (converted workshop content) * Advanced topics * Migration to npe2 * First generation plugins * Technical references * Plugin contributions reference * Plugin manifest reference * PyPI and napari hub metadata linking reference

How this proposal was developed

The proposed reorganisation and new pages have tried to follow the ideas from the Diátaxis framework, splitting documentation into Tutorials (step by step lessons or learning experiences), How-to guides (directions to solve specific problems), Explanations (understanding oriented discussions), and Reference (information-oriented technical descriptions) pages. The layout draws inspiration from the documentation of other open source projects with plugin or extension capabilities, napari plugin related content in external sites, and the recent updates to the napari.org site to use grid layouts on index pages.

This layout aims to achieve two main objectives:

Aim 1: align documentation with user/developer flow

---

Click to view personas and flow diagrams


---

The goal is to divide the documentation into pages that cater to either users or developers. We assume that plugin users want to discover, install, and effectively utilise plugins to solve their problems. On the other hand, plugin developers want to build and maintain plugins to support the users. The documentation should align with the typical flow for both users and developers, considering that these groups are not mutually exclusive. Additionally, this division between users and developers can also help describe when a user might consider becoming a plugin developer, how to do so, and what makes a great plugin. For example, this information would be useful if a user wants to share their napari script or widget with others or contribute to an existing plugin.

Aim 2: centralise plugin information

Currently, there is a wealth of information for plugin users and developers, but it is somewhat scattered. The goal is to centralise this information on the plugins site. Importantly, the aim is not to duplicate content on napari.org but to provide short summaries and cross-link to the relevant main information page. At the end of this issue, you'll find a non-exhaustive table which lists many of the current resources that are available for plugin developers and users.

The location of napari hub related documentation

The proposal includes incorporating some napari hub related documentation on the napari site. This is in an effort to encapsulate information in one highly discoverable documentation area, benefiting both the typical user journey, which often involves finding plugins on the napari hub, and the typical developer journey, which usually ends with publishing their package to PyPI and the napari hub. By placing napari hub related information on the napari site, it may make it clearer how to use the built in napari functionality (such as the plugin installer menu) in conjunction with or separately from the napari hub. Additionally, it may help clarify the usage of plugin metadata by napari itself, its usage by the napari hub, and the overlap between them.

However, very good cases can also be made for keeping napari.org/plugins completely separate from hub related information. In this case, perhaps that information could be placed on the napari hub and cross linked to from this documentation.

Implementation

The implementation of this proposal would involve:

1. A reorganisation of the existing plugin documentation into the new structure.
2. The creation of new documentation pages.

This process would likely occur in stages. The reorganisation of the existing documentation would be prioritised, allowing it to be available in the new structure as soon as possible. The consolidation of existing information would follow, and new pages would be created as time permits.

Related issues

There is one main open issue with some overall suggestions on how to structure and add to the plugin developer information (as opposed to a specific page change, for e.g.), which could be integrated as part of this effort #25. However, there are also many open issues related to specific changes to certain pages. These issues are not listed here to maintain focused on the overall structural improvements.

Resources for documentation

Here is a non-exhaustive list of resources that are currently available for plugin developers and users. These resources could all be valuable in the creation of the new plugin documentation.

Click to view the plugin documentation resources

| Category | Name | Location | Content | Target | | -------- | ----------------- | -------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------- | | General | Napari website | [https://napari.org/stable/](https://napari.org/stable/) | How to install and use Napari with examples. Community and contributing guide. API reference. In depth explanations. Roadmap and changelog. | user, developer | | General | Magicgui website | [https://napari.org/magicgui/](https://napari.org/magicgui/) | How to create automatic Qt widgets with API and examples. | user, developer | | Support | Image.sc Forum | [https://forum.image.sc/tag/napari](https://forum.image.sc/tag/napari) | General help with Napari for users and developers. | user, developer | | Support | Zulip chat | [https://napari.zulipchat.com](https://napari.zulipchat.com) | Developer focused help and discussion, more technical discussions. | developer | | Support | Email hub team | [team@napari-hub.org](mailto:team@napari-hub.org) | Contact the hub team directly for queries. | developer | | Plugins | Napari plugins | [https://napari.org/stable/plugins/](https://napari.org/stable/plugins/) | How to build a plugin, and what plugins can provide. How to test plugins. Difference between npe2 and v1 plugins. Note on installing plugins. | developer | | Plugins | Hub | [https://www.napari-hub.org/](https://www.napari-hub.org/) | Search for plugins and discover plugins. Link with PyPI for plugins. | user, developer | | Plugins | Hub FAQ | [https://www.napari-hub.org/faq](https://www.napari-hub.org/faq) | Address common questions around using napari, finding plugins, and building / sharing plugins. | user, developer | | Plugins | Hub GitHub | [https://github.com/chanzuckerberg/napari-hub/wiki](https://github.com/chanzuckerberg/napari-hub/wiki) | Guide to the hub team, and a developers guide to using the Napari hub. | developer | | Plugins | Cookiecutter | [https://github.com/napari/cookiecutter-napari-plugin](https://github.com/napari/cookiecutter-napari-plugin) | A template for creating new plugins for Napari following the suggested project structure. | developer | | Plugins | Hub preview | [https://github.com/apps/napari-hub](https://github.com/apps/napari-hub) | Preview a napari hub page before release to improve the listing. Integrates with GitHub. | developer | | Plugins | npe2 GitHub | [https://github.com/napari/npe2](https://github.com/napari/npe2) | Guide to convert old plugins to the new npe2 plugin format. Documentation that is pulled into the main napari site about what plugins can provide. | developer | | Plugins | Magicgui examples | [https://github.com/napari/magicgui/tree/main/examples](https://github.com/napari/magicgui/tree/main/examples) | Examples of how to use Magicgui widgets | developer | | Assorted | Napari videos | [youtube UCbTgw84ew4pxTJ9qu3W2hqg/playlists](https://www.youtube.com/channel/UCbTgw84ew4pxTJ9qu3W2hqg/playlists) | Examples of using Napari, workshop videos, demos of plugins in napari. Napari intro videos. | user, developer | | Assorted | CZI videos | [youtube ChanZuckerbergInitiative/playlists](https://www.youtube.com/c/ChanZuckerbergInitiative/playlists) | CZI lead efforts in napari plugin development. Videos from plugin groups and from plugin workshop. | developer | | Assorted | Workshops | [https://napari.org/stable/further-resources/napari-workshops.html](https://napari.org/stable/further-resources/napari-workshops.html) | Workshops with videos, slides, and other materials that do/may have info for plugin developers. For instance, the [accelerator workshop series](https://chanzuckerberg.github.io/napari-plugin-accel-workshops/) had information around npe2, plugin development tips, user experience, plugin usability, plugin reach, and plugin quality on hub. | developer |

[seankmartin]

One thing that would be nice to add into the publishing step is some information about the npe2api and how this relates to and forms the backbone of the napari plugin installer menu. For instance, that the installer menu is populated from the summary information for each plugin.

This could also be a good place to bring up bundled napari support - since I think (and please correct me if I have this wrong, I'm not 100% versed on the npe2api) that a napari plugin only supports bundled napari if it exists at https://npe2api.vercel.app/api/conda/{plugin-name} and the conda_platforms information matches your OS.

Also, it would seem to me that all of the plugins in npe2api errors.json can't be used in the napari viewer due to manifest (or other) errors? If that is correct - then we could point out to check this page to make sure your plugin is not there!

[psobolewskiPhD]

This looks outstanding! Wow! Thanks for putting the time into this. I think this is a great idea and a really important issue to adress.

One tiny note, but in that mockup I think it may worth emphasizing and splitting the paths for the two personas even more. I doubt there is any overlap between them in terms of content expectations. Maybe arrange the two persona tiles side by side? or using tabs like some of the other docs where we have pip and conda or something?

[seankmartin]

Possible PRs to restructure and add content to the plugin documentation

This is a possible set of PRs that might be required to achieve this restructure and addition of content. The PR groupings are suggested in the order shown below, but the PRs within each group can be tackled in any order. The main motivation for the groupings are in order to get some potentially high value PRs merged early on, and to provide reference material for later PRs. However, there should be no hard dependencies between PRs in different groups. The PRs are also linked to some existing issues that may be solved by that PR. At the end of this document is a visual representation of which parts of the restructure are being tackled by these PRs.

Proposed pull requests

Organize content

• [x] Reorganize the table of contents: Use the sections from the proposal. Currently, there is enough content to make the Building a plugin, testing plugins, advanced topics, and technical references sections. The other pages can standalone.
• [x] Rename some pages: As per the proposal.
• [x] Update the plugins landing page: Create a grid layout introduction, split into users and developers.

Plugin fundamentals

• [ ] What functionality plugins can provide: Explain from a user perspective what benefit plugins may give them, and what can/can't change (e.g. no new layers).
• [ ] Getting support for plugins: Explain that reaching out on Image.sc, zulipchat, or the plugin's github are usually the best way to ask for help and support when using plugins or installing plugins.
• [ ] How to use plugins: Demonstrate how to select plugins in the napari GUI to help solve a task, how to launch napari with plugins already loaded, and how to use the plugins from code.
• [ ] Plugin anatomy: What are the basic building blocks of a plugin? The minimal fundamental pieces required to label something as a napari plugin.
• [ ] Update the finding and installing plugins guide: This page is outdated due to the plugin installer changing, and the page could also be expanded. Could close: https://github.com/napari/docs/issues/122.
• [ ] Key concepts for plugins: Summarize some key points, such as multi-threading, and cross link to other sections of the documentation for more in depth-guides. Overall, some pointers as to what makes a great plugin. Try not to duplicate content here. This should also be distinct from best practices - where the best practices are more aimed towards 'hard' rules.

Update content

• [ ] Testing guidelines: Add information on napari fixtures that are relevant for plugins. Could close: https://github.com/napari/docs/issues/150.
• [ ] Best practices: Update the best practices with the key concepts page in mind. Could close: https://github.com/napari/docs/issues/131.

New guides

• [ ] User experience guidelines: Create a single page that addresses the elements of the suggested user experience section that are currently tenable. Focus on the technical aspects of creating a UI in line with guidelines.
• [ ] Describing and categorizing your plugin:  Non-hub specific explanation as to what parts of generic python package setup metadata (as used by PyPI and conda for example) are picked up by napari to display your plugin. Additionally how the https://github.com/napari/npe2api links into this. Finally, what difference publishing to PyPI and conda gives you.

New tutorials

• [ ] Using plugins to complete a task: Can demonstrate using a single (or a chain of) plugins to complete a task, such as cell segmentation. A few guides already exist like this, so we could convert that content if allowed. Or create some sort of abridged version and link to other guides. Related: https://github.com/napari/docs/issues/134.
• [ ] Converting a napari script into a plugin: When might I want to make my personal code into a plugin, what do I need to consider when I do this, and how do I do this?
• [ ] Building a plugin to complete a complex task: For instance, we could demonstrate the building of a cell segmentation plugin. Related: https://github.com/napari/docs/issues/57.

Hub related information

• [ ] Filtering plugins on the hub: How to use the hub to narrow down your plugin search. Can link out to information on the hub too.
• [ ] Setting up the napari hub preview page: How to create a preview of your plugin on the napari hub.
• [ ] Enhance your plugin's reach and demonstrate quality: How to support users and garner attention for your plugin, in addition to showing its quality.
• [ ] PyPI and hub metadata linking: How does the napari hub use standard plugin metadata, and where can you set your own plugin metadata.

Concerns

• Balance making information easy to find while trying to avoid duplicating content to cause a maintenance burden.
• The napari hub information may be desired to be on the napari hub itself or the napari hub GitHub wiki instead of on napari. In that case, it might be beneficial to create these pages on the hub side, and then provide a summary of napari hub functionality either on napari itself or on the hub. Regardless, this information should hopefully be linked pretty early on in the user / developer journey with plugins - as the hub is a pretty integral part of the napari plugin ecosystem.
• Since the overall effort may take a while here, a potential high value effort could be to link out to external documentation with plugin related elements.

A visual representation of the pull requests

Click to view the diagram


cc @DragaDoncila @dgmccart

[psobolewskiPhD]

One thing that came to mind is explaining/contrasting plugins vs. widgets. A plugin can have a widget, but you can also make widgets without a plugin.

[seankmartin]

Tracking related PRs:

• Restructure at https://github.com/napari/docs/pull/203

[lucyleeow]

239 could fall under this body of work.