Documentation Reorg: How-to and reference guides

[philippjfr]

We are currently in the process of reorganizing our documentation and one of the central themes is to separate explanations, how-to guides and reference material. Our current user guides are a total mish-mash of all of these and it's quite awkward to find what you want. So my first goal is to come up with a list of How-to guides, additional reference guides and sections on where to put them. Please chime in with additional suggestions.

How To Guides

APIs

• A primer on using Param.
• How to use pn.bind?
• How to use Parameterized classes and pn.depends?
• How to use and set up callbacks?
• How to use interact?

Customization & Layout

• How do I lay out components?
• How do I customize the look and feel of a component?

Templates

• How do I use a default template?
• How do I create a custom template?

Editors and notebooks

• How do I use Panel effectively in a notebook?
• How do I use Panel effectively in an editor/IDE?

Advanced features

• How do I display notifications?
• How do I perform deep linking?
• How do I build a REST API?
• How do I schedule tasks?
• How do I cache data across sessions?

Server deployment

• How do I configure a Panel server?
• How do I configure OAuth?
• How do I add additional endpoints to my server?

Reference Guides

• panel serve
• pn.serve
• pn.bind
• pn.config
• pn.state
• ...

[MarcSkovMadsen]

Hi @philippjfr . I love how-to guides. I was thinking that your how-to titles are technical and solutions. Users might not know they should be interested in pn.bind as a solution to their problem. When the search they only have a rough idea of their problem. In the case of pn.bind they would like to make a python object like a plot depend on the values of widgets.

So I was thinking the the titles should be more like How to solve problem X using the solution Y.

So for example instead of how to use pn.bind it should be *How to make a plot interactive by using pn.bind to bind functions to widgets. This will help the user to understand when pn.bind is the right solution. C.f. Sophias recent blog post on the 3 apis.

Looking at the Diataxis frameworks description of Howto also makes it clear that they provide solutions to user problems. And they mention recipes: When you search for recipes you search for the what (pan cakes) not the how (use a pan).

.

So for example the Howtos on APIs could be

APIs

How to quickly explore your data interactively by using pn.interact How to build an interactive component by using pn.bind How to build reusable and composable components by using Param. How to add a file download button by using a callback

(I would remove A primer on using Param. as its a tutorial).

[philippjfr]

Excellent points @MarcSkovMadsen, I agree with everything you said.

[MarcSkovMadsen]

An idea could be to let 5 of us write one how to each. We would probably get 5 different ways of doing that. From that try to discuss, align and agree on how our how to should be. In that way we would have 5 persons that feel ownership of how tos. And writing one how to should not be that big an effort :-) They should be adressing one problem, short and to the point similarly to a recipe.

[MarcSkovMadsen]

And there is a reward at the end of the tunnel

[MarcSkovMadsen]

For me how-to should take a lot of the look and feel of blog posts. They feel modern and fun. They solve a specific problem. The most useful blog posts are easiy to follow and can easily be transformed (copy pasted or downloaded) into a starting point

[MarcSkovMadsen]

Two examples to the How to list

• How to convert a Streamlit data app to a Panel data app
• How to convert a Dash data app to a Panel data app

[MarcSkovMadsen]

The new pyodide powered documentation is awesome. Everybody would like to be able to do that. I think we should add a how-to guide on that

• How to create a beautiful, interactive and shareable document using markdown and panel.

[droumis]

How to quickly explore your data interactively by using pn.interact How to build an interactive component by using pn.bind How to build reusable and composable components by using Param.

This is a improvement on the titles but given their wide scope (especially of the first - 'exploring data') and that APIs is probably the most important aspect of the documentation, I think we should still be careful about not having the content veer into mish-mash territory. How-to guides should be useful to people that already have familiarity and competence with Panel, are in their working phase, and just need a recipe to accomplish a specific problem that they are aware of. So (eventually) drilling down toward even more specific How-to tasks within each API seems like a good diataxis-y direction.

Maybe writing an Explanation type page about APIs in parallel, which can be linked to, will make it easier to omit the discussion of concepts and comparisons within the How-to type guides.

[MarcSkovMadsen]

A lot of inspiration for valuable topics for How-to guides can also be found in the posts on Discourse.

[philippjfr]

Just to say that I've decided to merge my PR starting the migration from user guides to how-to guides. Overall I'm quite happy with the way this has gone however I almost certainly carried over a bunch of material that is not in the "recipe" style so there will be a process in cleaning that up. There are also still a few user guides left over that have not been migrated anywhere. However overall I'm confident that this is a big improvement and the remaining material can be migrated into reference and background material.

The next step here is that @droumis will report back and suggest what to do with the remaining material.

Existing User Guides

Existing user guides were grouped

Core

• [x] Overview: This is largely reference material, providing a high-level explanation of the component types (widgets, panes, layout), a summary of the most commonly used methods and a description of the pn.config and pn.state classes.
• [x] Components: Again this is really reference material, providing a high-level description of the three component types.
• [x] APIs: Goes through each API and builds the same application with it. Can easily be turned into a number of smaller API how-to guides.

Reference

• [x] Customization: A horrible mix of reference material (e.g. describing common parameters like margin or background) and how to material (when to use which sizing mode)
• [x] Interact: A direct copy of the ipywidgets interact docs that doesn't really fit anywhere.
• [x] Widgets: A silly overview of widgets and a rough grouping into input, range and select widgets. No idea what to do with this.
• [x] Parameters: A mish-mash explaining Parameterized -> Widget mappings, ParamMethod/ParamFunction and then various how-to examples on how to leverage these. Can at least partially be migrated to how-to materials.
• [x] Linking: More mish-mash demonstrating Python and JS based linking. Some of it can become how-to material.
• [x] Templates: Reference material plus some how-to examples.
• [x] Pipelines: Reference material with one or two how-to examples
• [ ] Performance, Profiling and Debugging: Total mishmash of some performance tips, admin dashboard explanations etc. Can probably all become how-to material.

State, Caching & Callbacks

• [x] Session State and Callbacks: Converted to Session state and Callbacks how-to guides.
• [ ] Asynchronous and Concurrent Processing: Started converting the async section to a Async Callbacks how-to guide. Rest should become concurrency how-to section.

Export

• [ ] Display and Export: Partially moved into export sections, display sections are closer to reference material but maybe could be rewritten to be more how-to like.
• [x] Running Panel in the Browser with WASM: Migrated entirely to how to guides.

Server usage

Migrated cleanly to server configuration, server integrations, deployment and authentication how to guides.

Sections

• [ ] APIs: How to use the lower and higher level APIs in Panel
  • [x] Reactive Functions: Linking functions or methods to widgets using pn.bind or the equivalent pn.depends decorator.
  • [x] Interact Functions: Auto-generates a full UI (including widgets) given a function.
  • [ ] Parameterized classes: Declare parameters and their ranges in Parameterized classes, then get GUIs (and value checking!) for free.
  • [ ] Callbacks: Generate a UI by manually declaring callbacks that update panels or panes.
• [ ] Using Param with Panel: Discover how to use Parameters with Panel.
  • [x] Building UIs using Param: Discover how to generate UIs from Parameterized classes without writing any GUI related code.
  • [x] Declare Custom Widgets: Discover how to extend Param based UIs with custom widgets.
  • [x] Declare Parameter dependencies: Discover how to leverage @param.depends to express dependencies and trigger events based on UI interactions.
  • [x] Param subobjects: Discover how to structure Parameterized classes with subobjects to create nested UIs automatically.
• [x] Linking Parameters: Discover different ways of linking parameters in Python and Javascript.
  • [x] Watchers: Discover how to use the powerful but low-level .param.watch API provided by param to trigger callbacks on parameters.
  • [x] Links in Python: Discover how to use the convenient, high-level .link API to link parameters in Python.
  • [x] Links in Javascript: Discover how to use the convenient, high-level .jslink API to link parameters in Javascript.
  • [x] Link plots in Javascript: Discover how to use .jslink to link Bokeh and HoloViews plot parameters in Javascript.
  • [x] Javascript callbacks: Discover how to use the .jscallback API to write arbitrary JS callbacks linking one or more components.
• [x] Session callbacks and events: How to set up callbacks on session related events and periodic tasks.
  • [x] Asynchronous Callbacks: How to leverage asynchronous callbacks to run I/O bound tasks in parallel.
  • [x] Load callbacks: How to set up callbacks to defer a task until the application is loaded.
  • [x] Periodic Callbacks: How to set up per-session callbacks that run periodically.
  • [x] Session Callbacks: How to set up callbacks when a session is created and destroyed.
  • [x] Schedule tasks: How to schedule tasks that run independently of any user visiting the application(s).
  • [x] Bokeh server callbacks: How to safely modify Bokeh models to avoid running into issues with the Bokeh Document lock.
• [x] Accessing session state: How to access state related to the user session, HTTP request and URL arguments.
  • [x] HTTP Request: How to access information about the HTTP request associated with a session.
  • [x] URL state: How to access and manipulate the URL.
  • [x] Busy: How to access the busy state. (Maybe move to callbacks)
• [x] Caching: How to cache data across sessions and memoize the output of functions.
  • [x] Manual caching: How to manually cache data and objects on pn.state.cache.
  • [x] Memoization: How to use the panel.cache decorator to memoize (i.e. cache the output of) functions automatically.
• [ ] Display and Preview output: How to display Panel components and apps in your favorite notebook or editor environment.
• [ ] Exporting and Saving output: How to export and save Panel applications as static files.
  • [ ] Embedding Output: How to embed the application state to share simple applications as a static file.
  • [ ] Saving output: How to save Panel applications to HTML or PNG.
  • [ ] Bokeh model: Discover how to access the Bokeh model underlying a Panel component or application
• [x] Running in WebAssembly: How to run Panel applications entirely in the browser using WebAssembly, Pyodide and PyScript.
  • [x] Convert to WASM: Discover how to convert existing Panel applications to WebAssembly.
  • [x] Use from WASM: Discover how to set up and use Panel from Pyodide and PyScript.
  • [x] Sphinx Integration: Discover how to integrate live Panel components in your Sphinx based documentation.
  • [x] JupyterLite: Discover how to set up a JupyterLite deployment capable of rendering interactive Panel output.
• [x] Server Configuration: How to configure the Panel server.
  • [x] Launch from the commandline: Discover how to launch and configure a Panel application from the commandline.
  • [x] Launch programmatically: Discover how to launch and configure a Panel application programmatically.
  • [ ] Setting up a (reverse) proxy: Discover how-to configure a reverse proxy to scale your deployment.
  • [x] Access via SSH: Discover how to access a Panel deployment running remotely via SSH.
  • [x] Serving Static Files: Discover how to serve static files alongside your Panel application(s).
• [ ] Server Integrations: How to integrate Panel in other application based on Flask, FastAPI or Django.
  • [ ] Flask: Discover to run Panel applications alongside an existing Flask server.
  • [x] FastAPI: Discover to run Panel applications alongside an existing FastAPI server.
  • [x] Django: Discover to run Panel applications on a Django server (replacing the standard Tornado based server).
• [ ] Deploying applications: How to deploy Panel applications to various cloud providers (e.g. Azure, GCP, AWS etc.)
  • [ ] Azure
  • [ ] Binder
  • [ ] Google Cloud
  • [ ] Heroku
  • [ ] Hugging Face
• [x] Authentication: How to configure OAuth to add authentication to a server deployment.
  • [x] Configuring OAuth: Discover how to configure OAuth from the commandline.
  • [x] OAuth Providers: A list of OAuth providers and how to configure them.
  • [x] User Information: Discover how to make use of the user information and access tokens returned by the OAuth provider.

[droumis]

This is what I was drafting to comment in #4251, but it got merged already, so I'll leave the comment here:

In general, to rein in this massive PR a bit (referring to #4251), I would err on the side of getting the new structure in place, and then continue wordsmithing in future and more contained PRs. That said, you requested that I focus on what to do with certain user guide pages, so I will use one of them - Components - as a concrete example of how I’m thinking about this migration to a Diataxis-like architecture. Once we align on the approach, I will start helping to act on it.

1. The Components User Guide, as currently written, can essentially be used to initialize a Components page in the Background section. It is primarily a discussion about the concept of components and types of components, such as that they are built on the Param library and the distinguishing aspects of the different component types - Pane, Widget, and Panel (Row, Column, Tabs, GridSpec). The bulk of this content is understanding-oriented, which is most useful for the users in a learning phase rather than those in midst of working. Later on, further content can expand this proposed Components Background page, such as with design decision reasoning, technical constraints, or analogies/connections to other external/internal things.

2. Consider cherrypicking any crucial/core concepts that every new user should know and incorporate them into the Getting Started - Core Concepts page. The Core Concepts page currently contains some information about components; mainly in the ‘Display and Rendering’ and ‘Widgets’ subsections. While these sections of the Core Concepts page can be improved on clarity, I don’t yet see any obviously missing crucial concepts that could be derived from the Components User Guide page. Perhaps one candidate missing topic is the explicit linking of components, as an example of using .jslink is in the current Components page. But honestly, I don’t think that .jslink qualifies as a core concept. If you want something about explicit component linking in the Core Concepts, it should probably be .link which would come when parsing/migrating the Linking User Guide, so no action on this is needed now. I think the only addition to be made to the Core Concepts presently is a link to the Components page within the appropriate context. For instance, after the ‘explicit’ construction of pane type is mentioned in the Core Concepts page, I would add a mention/link to the ‘Panes’ section of the Components page.

3. Identify any useful ‘how to …’ questions that this page addresses. As a reminder, some characteristics of a Diataxis How-To page include having a specific scope, describing the problem, excluding discussion, including a sequence of actions, and following a consistent format. With that in mind, some simple candidate How-To pages that the Components User Guide could address include: how to load the Panel extension in a notebook, how to construct a pane, how to view pane type, how to access widget values, how to add widget and pane components to a panel, how to add a title to a tab panel, how to set a GridSpec panel to have responsive or fixed sizing, how to place an object at a specific location in a GridSpec panel.

4. I don’t immediately see how any of the Component User Guide might fit within a reference section. In general, the Reference Gallery and Reference API are still confusing sections to me and I think they should be handled outside of this PR. Perhaps, some simple examples from the Components page could be used within a reference, but these seem to already be covered.

[droumis]

Just noting that I spoke to @philippjfr and he agreed with my approach outlined above.

[philippjfr]

Closing, the how-to guide will always be evolving but @droumis and I made great progress here.