Ideas for improving the new documentation

[MarcSkovMadsen]

The upcoming revised documentation is a great improvement. I've already learned a lot and believe I understand Param and the ideas behind it much better. It's great.

The below is a list of ideas that I believe could improve the documentation for both new and experienced users.

1. Make it even more visible WHY Param would be valuable to "me".

The documentation is text heavy. I think that is great as soon as my attention has been caught and I understand why I should take the time to read and learn. The text is great.

I need something to catch my attention though. And explain it to people with other learning preferences than text.

My suggestion would be to add a 3 min video to the Introduction that explains the why with a real world, practical example. It should be a walk through of a very interactive Notebook that can be downloaded by the user right next to the video. And add in a link to Binder where I can try it out as well.

It might also be valuable to add a 30 sec .gif video that just plays and immediately explains the Why to me and catches my attention.

2. Make Getting Started Example even more friendly for users.

The first example I see looks like this.

import param

class A(param.Parameterized):
    title = param.String(default="sum", doc="Title for the result")

class B(A):
    a = param.Integer(2, bounds=(0,10), doc="First addend")
    b = param.Integer(3, bounds=(0,10), doc="Second addend")

    def __call__(self):
        return self.title + ": " + str(self.a + self.b)

>> o1 = B(b=4, title="Sum")
>> o1.a = 5
>> o1()
'Sum: 9'

The main question for me is whether using __call__ in the example is a good idea. I know about __call__ but I never see any colleagues or blogs in data, engineering, trading or science use it.

So I believe the big questions for users is: What is __call__. Why do you use __call__. I did not know I needed __call__ in my code. What is the use case for __call__? Why is __call__ useful to me?

So unless there is a point (which a have not yet understood) in using __call__ I would rename the function to sum.

You are also using multiple inheritance which might also throw some users off. I just see my new colleagues coming out of engineering and finance education struggling with the concept of Classes.

3. Add Pydantic to the Comparison

Pydantic is taking the Python world with storm. Together with FastApi. I would like to know why it's valuable to me to build domain models with Param when

1. I can use Pydantic and not Param with FastAPI.
2. Pydantic works using type annotations which provides a much better development experience in an IDE and with linters than Param.
3. The huge popularity of Pydantic means more colleagues would probably know and understand why I use Pydantic
4. There are (more) tools out there for Pydantic like PyCharm, MyPy, Hypothesis plugins. And more are probably coming.
5. Lots of orgs are building on Pydantic and want to be associated with it. See https://github.com/Kludex/awesome-pydantic

I believe Param and Pydantic could be integrated and that would help a lot of Pydantic users easily create visualizations, tools, apps and dashboards from their models. And Param users create modern Rest Apis via Fast Api. For a Streamlit+Fast API example check out https://testdriven.io/blog/fastapi-streamlit/.

4. Add a real world example

Add a section with a real world example. My personal profile would simply need that. You can write just as many theoretical arguments about why Param is great (which it is) but without a real, practical example you don't catch my attention. I move on.

I work with traders so in my world the Black Scholes option pricing model is a great example.

I would visualize it using Panel.

There are probably more generally understandable examples out there but this one I use at work to explain why Param is powerful.

FYI. @jbednar

[jbednar]

1. Make it even more visible WHY Param would be valuable to "me". My suggestion would be to add a 3 min video to the Introduction that explains the why with a real world, practical example. It should be a walk through of a very interactive Notebook that can be downloaded by the user right next to the video. And add in a link to Binder where I can try it out as well.

Sounds like a great idea!

However, what do you mean by "interactive"? If you mean Panel, I think that may muddy the waters a bit; I think it's important for people to understand that Param is important in its own right, whether or not you ever make a GUI with Panel. So I think any first example should not use Panel.

2. Make Getting Started Example even more friendly for users. def __call__(self): The main question for me is whether using call in the example is a good idea. I know about call but I never see any colleagues or blogs in data, engineering, trading or science use it. So I believe the big questions for users is: What is call. Why do you use call. I did not know I needed call in my code. What is the use case for call? Why is call useful to me?

Hmm; good point; probably assuming people know too much about Python classes. Personally, I believe that if a Python class does only one thing, as this one does, it's simply good practice to name that thing __call__, instead of coming up with some other arbitrary method name for people to learn and keep track of. I.e., __call__ just means "do the thing this class does, whatever that is". If all other things are equal, I believe examples should demonstrate good practice by default, to maximize the exposure that people get to good patterns rather than bad patterns.

Still, it's also important not to derail people from their current objective by confusing them, particularly not in introductory examples; it's not our job to train people in Python language feature usage! So I'd be happy to accept a change to rename that to sum, to avoid putting off people without that background.

You are also using multiple inheritance which might also throw some users off.

This example is not using multiple inheritance. Each class inherits from only a single base class, i.e. B(A) (B inherits from A) where A(param.Parameterized) (A inherits from Parameterized).

But sure, you're asking whether it's important to show such an inheritance hierarchy here. I do think inheritance hierarchies like that are an extremely important aspect of Param, and I do want all Param users to be aware that Param is particularly well suited to such hierarchies, allowing you to avoid duplication along the inheritance chain (particularly for docstrings). Without Param, tall inheritance chains almost never bother to document individual parameters well, because that same bit of docs would have to be duplicated at every parent and child class in the chain, because Python inherits methods and attributes across the hierarchy, but not attribute docs. If we don't have such a chain in the examples, we can't show one of the main reasons to use Param, i.e. that it lets you organize your code well and without duplication. Without such inheritance, Python users have a disincentive to design proper inheritance hierarchies; Param fixes that!

That said, it doesn't necessarily need to be the first example! :-)

3. Add Pydantic to the Comparison. Pydantic is taking the Python world with storm. Together with FastApi. I would like to know why it's valuable to me to build domain models with Param when

Great idea; can you please contribute a section covering Pydantic to the Comparison? I don't use Pydantic myself, and from a relatively brief study of the docs I don't see what it would offer me that I don't already have with Param. It would be great to have a more useful perspective from someone who has used both!

I believe Param and Pydantic could be integrated and that would help a lot of Pydantic users easily create visualizations, tools, apps and dashboards from their models. And Param users create modern Rest Apis via Fast Api. For a Streamlit+Fast API example check out https://testdriven.io/blog/fastapi-streamlit/.

I can see how Pydantic could be supported by Panel. I.e., just as Panel already does for Param, Panel could accept a Pydantic BaseModel-derived class and read the metadata it needs to synthesize widgets, etc. from it. I'm not quite seeing how Param and Pydantic could interact, though; can you explain? Seems to me like it would be confusing to mix Param and Pydantic in a particular class, even if you prefer Pydantic's syntax for specifying types.

4. Add a real world example. Add a section with a real world example. My personal profile would simply need that. I work with traders so in my world the Black Scholes option pricing model is a great example.

Maybe we'd want two very different real-world examples? Black Scholes is fine as one of them, as long as it's a simple Param class that works well with Panel.

To-do items:

• [x] Make a 2-5 min video that explains the "why" with a real world, practical example. It should be a walk through of a very interactive Notebook that can be downloaded by the user right next to the video, plus a link to Binder for trying it out.
• [x] Link to the video from the introduction and probably home page.
• [ ] Make a 30-sec .gif video that just plays and immediately explains the Why to me and catches my attention.
• [ ] Link to the gif from the introduction and probably home page.
• [ ] Change __call__ to sum
• [ ] Consider whether to put a first example with only a single level of inheritance
• [ ] Add pydantic to Comparisons
• [ ] Consider adding Pydantic(PaneBase) to Panel, analogous to Param(PaneBase).

[MarcSkovMadsen]

Here is a video illustrating the concept.

https://user-images.githubusercontent.com/42288570/122000237-c5b66280-cdae-11eb-8eee-a3d33c1d0b53.mp4

[MarcSkovMadsen]

and here is a faster version

https://user-images.githubusercontent.com/42288570/122000307-e383c780-cdae-11eb-9751-1f152243b0e0.mp4

[MarcSkovMadsen]

I will make a PR with the notebook. But please consider giving me right to push to the repository like I have for Panel @jbednar .

[ea42gh]

I'd be happy to edit such a video!

The way I see it, this would involve 1) creating the notebook I advocate using folding for each of the sections: this would require each of the section/subsection titles be in their own markdown cells 2) multiple recordings of somebody explaining the notebook. Here, audio quality is the most important consideration. 3) I could then take these multiple takes, and edit them to create a video.

How would we get this organized?

[jbednar]

Cool! Yes, please do submit the notebook, e.g. as examples/intro.ipynb or examples/promo.ipynb and we can edit that until we're happy with it. Then we can record something with audio and go from there (and address the other to-do items here).

[MarcSkovMadsen]

I will also submit the Notebook ASAP and we can iterate on it. But maybe it would be an idea to write down and formulate the purpose and target audience of the material. That could guide us when discussing content, layout, format etc.

For inspiration on what @ea42gh can do with video check out this video that @ea42gh produced for panel-chemistry https://drive.protonmail.com/urls/0XQCDYYY84#Zd1ZRPWvIg2X

[MarcSkovMadsen]

Started a Promo Notebook PR here https://github.com/holoviz/param/pull/487. Feel free to comment or contribute.

[ea42gh]

I have some suggestions for the documentation I'd like to make since we want to produce videos.

E.g., using <br>$\quad$ to reduce line lengths, adding additional subsections to create more of a framework for the audio, changing font sizes.

What is a good way to cooperate on this? PR on top of PR seems a bit clumsy!

[philippjfr]

@ea42gh Happy to make you an external collaborator so you can push to @MarcSkovMadsen's PR.

[ea42gh]

@phillipjfr How does that work? Can you point me to some explanation I could read? Does this cover it? https://medium.com/google-cloud/pull-push-to-someone-elses-upstream-github-pr-6073ae5005e7

[MarcSkovMadsen]

I am perfectly confident you doing an iteration on the "draft version" I made.

For me the purpose is to make sure

1. Not so strong Pythonistas understand if Param can be useful for them and is worth their effort to learn.
2. Strong Pythonistas immediately understand what Param is about and understanding enough to actually start using it.

So anything that can make the notebook communicate better is key.

[philippjfr]

@ea42gh Now that I've added you (and once you accept), you will be able to clone the repo, check out Marc's branch (once he's pushed it to the param repo), and push directly to that.

[ea42gh]

@jbednar re audio/video, some thought should be given to consistent branding:

1) intro and outro visuals and possibly background music 2) color theme for annotations (bubbles, arrows, etc; design should yield good readability for text over background) 3) a consistent set of annotations and transitions

What are your thoughts?

[MarcSkovMadsen]

Opened new and revised PR here https://github.com/holoviz/param/pull/488

[ea42gh]

I pushed a new version of Promo.ipynb to the param-promo branch and forgot to clear the outputs first :(

The idea is to have all sections folded prior to recording a video, then open each section/subsection in turn and execute code, if any.

[ea42gh]

The panel_chemistry video @MarcSkovMadsen mentioned above has been replaced with a newer version: https://drive.protonmail.com/urls/E7GSH67AX8#AXEjIroNYA6i

[ea42gh]

To push this forward, I created a video from the Promo notebook. Everybody interested (@philippjfr @jbednar @MarcSkovMadsen ?) please comment!

https://drive.protonmail.com/urls/K66V127PX8#RacCEK6TjoXa

Some of my own notes:

• audio quality is extremely important: I did not get clean audio when I recorded scenes
• The timing is off in places. Easy to fix, but I did not spend time on a placeholder
• I did not use any highlighting, zooming, panning for the same reason... note that for the highlighting to work well, a light background is preferable!
• I noticed that the parameter documentation strings and compute_hypothenuse should be corrected and anglicized...

[jbednar]

Thanks! I'm on spotty internet until Wednesday, but will take a look then.

[ea42gh]

Hope that means a great vacation!!

[rtbs-dev]

Just finding this by searching "pydantic" on the param issues list! Glad I'm not the only one!

I can see how Pydantic could be supported by Panel. I.e., just as Panel already does for Param, Panel could accept a Pydantic BaseModel-derived class and read the metadata it needs to synthesize widgets, etc. from it. I'm not quite seeing how Param and Pydantic could interact, though; can you explain? Seems to me like it would be confusing to mix Param and Pydantic in a particular class, even if you prefer Pydantic's syntax for specifying types.

Definitely see this, though it comes down to what kind of "support" you guys want to be thinking about. It would 100% make me happy to have a Pydantic(PaneBase). Though this might require more maintenance on your end? Two separate interfaces? But I suppose the primary alternative is to make a (round-trip-able?) conversion between param and pydantic, so that pydantic models can be used to generate Param's incrementally, as the feature gets supported?

Either way I have a bunch of pydantic models that I'd be willing to test out with panel when the time comes. This would let procedural UI creation from e.g. JsonSchema's be way easier!

Also probably worth checking out Pandera, which may be even closer to the way pandas get's used in Holoviews, but is increasingly interoperable with Pydantic. Hypothesis property testing for my dashboards, anyone? :smile:

[jbednar]

Thanks, @tbsexton , for making the possible Pydantic link more concrete. If indeed what you'd like is something acting like Pydantic(PaneBase) for Panel, I agree that at that point it's an open question whether to achieve that by directly implementing Pydantic(PaneBase) analogously to Param(PaneBase) (in Panel), or whether either a one-way or two-way converter between pydantic.BaseModel and param.Parameterized would be easier or work better (implemented using Param mechanisms alone).

I.e., whether or not Param itself makes sense to use with Pydantic, it does make sense to consider whether we could use mechanisms at the Param level to make Pydantic be well supported at the Panel level. (Was there some reason these all had to start with "P"??) I think this is concrete enough for me to discuss with @philippjfr and see if there's an obvious way forward for working with Pydantic in Panel. But I do think he and I already have our plates quite full for the next month at least, so if anyone else wants to try making some sort of wrapper, that's probably the only way it would move forward in the immediate future.

[jbednar]

@ea42gh , thanks so much for the video! I made changes to the notebook in https://github.com/holoviz/param/pull/488 , but I'll list comments specific to the video here. Basically, it looks great to me! Some minor notes:

1. At 8 minutes, I think it's a bit long for such a video. I think we have about 5 minutes of the user's attention, at most. Maybe cut the explicit event watching?
2. The music helps make it seem more professional, but on balance I think I'd rather just launch right into it, with no intro or outro music. This is some hands-on stuff, and I think people will want to jump right in!
3. In one bit you say "We have to initialize our class, of course", but I'd be careful there; it's true that the class has to be initialized, but it's only rarely required for a user to supply an explicit constructor; the default constructor that processes keyword arguments is very often the only constructor needed. Obviously we can't get into all that here, and I don't see a way to avoid having the explicit constructor in this case, so maybe just don't say "of course". :-)

Other than that, if people are happy with the notebook, I'm happy for you to record a slightly updated version and we can use that! Thanks so much.

[ea42gh]

@jbednar do you really want me to voice the video? I did not contribute to param, and I do have an accent!

I have always had trouble with how to start: @MarcSkovMadsen's "Hi there!" seems to work?

I think future videos should have a consistent look and feel, so please feel very free commenting and requesting changes. And yes, it is trivial to cut out words and sentences, and replace the visuals....

Unless I hear from you, I'll record a new version for further comments.

[jbednar]

I don't mind the accent, and this is a contribution to Param! :-)

"Hi there!" is certainly fine. I think future videos can have a variety of authors and speakers, to show the diversity and widespread applicability of Param!

[ea42gh]

@jbednar @MarcSkovMadsen I put up two new versions of the promo video:

1) the original file: https://drive.protonmail.com/urls/SAVKC3CWP0.#2foouSqCo1qY 2) the file with highlighting zoom and pan. https://drive.protonmail.com/urls/F3SK7AEPG4#CJ5JZ6fNo8Ds

please give me comments, change requests, etc..

[MarcSkovMadsen]

Hi @ea42gh . I took a look at video 2. I think its close to perfect. The two comments I have are

1. Personally I miss the intro music. I think it sets a nice inviting and relaxing scene for me to listen and learn. And just gives the video that extra professionalism other videos do not have. Without the music the video gets much more serious over fun.
2. I hear one or more birds in the background by the end of the video. Maybe they should be removed?

[ea42gh]

I Hi @MarcSkovMadsen

• Re 1: you and @jbednar are on opposite sides for this one! Personally, I like an intro just short enough to read a title for a series of videos: a unified look and feel. (Hint, hint, shall we do more videos?)
• Re 2: ouch, dubbing in audio is difficult: without a professional sound studio, the audio recordings never sound the same. I'll go see if I have some extra material to dub in! Just when are these bird sounds? I just listened to the video 3 times, I guess I get distracted by my voice...

[MarcSkovMadsen]

The bird is there in file https://drive.protonmail.com/urls/F3SK7AEPG4#CJ5JZ6fNo8Ds around 4:14.

I don't think it's a big problem.

[ea42gh]

@MarcSkovMadsen I had audio I could use: this version does not have a bird around 4:14 https://drive.protonmail.com/urls/SV5465BPMC#Ye4coiEhfEI7

[rtbs-dev]

I.e., whether or not Param itself makes sense to use with Pydantic, it does make sense to consider whether we could use mechanisms at the Param level to make Pydantic be well supported at the Panel level. (Was there some reason these all had to start with "P"??) I think this is concrete enough for me to discuss with @philippjfr and see if there's an obvious way forward for working with Pydantic in Panel. But I do think he and I already have our plates quite full for the next month at least, so if anyone else wants to try making some sort of wrapper, that's probably the only way it would move forward in the immediate future.

So I might be possibly interested, but I don't have too much insight into how panel grabs the Param objects in the first place. I assume the relevant code is in this file? So you imagine making a new version of this whole class for something like a pandera or pydantic class?

I imagine some issues down the line when e.g. other projects that have the "same shape" as pydantic come along (e.g. for xsd files and we have to once again do this "mapping". I would imagine a possible way to avoid repeating the "n-to-m parser problem" is to have some kind of wrapper-class-generator.... you give me a list of object/type-mappings that tell me what objects make what Panel/Param objects, and I will give you a wrapper class (or maybe just a nice Param instance in the first place).

[ea42gh]

@jbednar could you let me know what changes you would like for https://drive.protonmail.com/urls/SV5465BPMC#Ye4coiEhfEI7 my window will close next week for the rest of the month...

[MarcSkovMadsen]

@jbednar . Some guidance on where to store it and how to get it into the documentation of Param would be needed. Thx.

[jbednar]

@ea42gh , that looks fabulous! I'm happy to use it as is, but I did push a tiny change to the Promo.ipynb just now, replacing the github repo URL github.com/holoviz/param with param.holoviz.org as what we point people to. That comes up in the very last bit of your video, and I'd rather point them to the real website instead of github, which should go live this weekend. If it's a lot of trouble to update that one little bit, don't bother, but if it's simple, sending people to the public site would be best!

As for where to store it, I'd just upload it to YouTube if no one objects. My plan is to link it to the very first line of the Param site:

# Welcome to Param!

<h1><img src="_static/logo_stacked.png" width="125"></h1>

Are you a Python programmer? If so, you need Param, and check out our 
<a href="https://www.youtube.com/watch?v=0oRBvnCiUJE">8-min intro video</a> to see why!

(replacing the link once I upload it).

[ea42gh]

@jbednar @MarcSkovMadsen Done. I also tweaked the highlights a little bit. This new version is at https://drive.protonmail.com/urls/CX37TF12M8#hoALmTcHNNpj

I have no problem seeing it on youtube. We all have channels by now!

[jbednar]

Ok, it's up at https://youtu.be/KP9bRmzinaY and will be linked as soon as the release is final. Thanks so much to both of you!

[ea42gh]

@jbednar One small comment on the description of the video on youtube: it mentions the 'param library', but the description of what it is is hidden in the 'show more' section. May be turn this around a bit?

[jbednar]

Good idea. Done!

[ea42gh]

@jbednar, the description of what param does is hidden in the 'show more' section. I don't need to see my name attached to the video, but a reader should immediately see what param does...

[jbednar]

Good point! Also now done!

[MarcSkovMadsen]

Just a few suggestions to improve the youtube title and description @jbednar

Title: Either "Param: Powerful Parameters for Python Applications", "Param: Parameters for your Python Applications" or "Param: Powerful Parameters for your Python Applications".

For me adding "Powerful" and/ or "your" is more "catchy".

Description "Param (https://param.holoviz.org) is a library providing Parameters: Python attributes extended to have features such as type and range checking, dynamically generated values, documentation strings, default values, events etc. Param enable you to create more robust and powerful applications in fewer lines of code."

I would add the last bit so that it is shown to users why parameters are useful to them and why they should check out the video and the library.

[MarcSkovMadsen]

One more suggestion @jbednar . Instead of only a link to the video. I would display the video.

For example via something like

<div align="center" style="margin-right:10% margin-left:10%;">
    <iframe width="100%" height="400" src="https://www.youtube.com/embed/KP9bRmzinaY" title="Param: Python Parameters" frameborder="0" allow="accelerometer; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
</div>

Alternative a screenshot of the video with a link to youtube on top.

It would make the video more attractive and also add something visual to the Param web site.

[MarcSkovMadsen]

And add a link to the promo.ipynb notebook both in the youtube description and just under the video on the web site. Potential users would like to follow along and try it themselves. At least that is what I would expect.

[jbednar]

Ok, I've done all that in https://github.com/holoviz/param/pull/502, though really I'm just trying to get something to go live here! :-) The main thing left is getting the reference manual organization ok, and once that's done, I'll tag the release. We can make any other changes later.

[jbednar]

@tbsexton,

So I might be possibly interested, but I don't have too much insight into how panel grabs the Param objects in the first place. I assume the relevant code is in this file? So you imagine making a new version of this whole class for something like a pandera or pydantic class?

Yes, that's what I was imagining, if Pydantic support were added at the Panel level. Well, ok, first the panel.Param class would need to be split so that much of it goes into a superclass called something like panel.DeclarativeObjectWrapper, containing all the general-purpose code that's about discovering and formatting widgets backed by a declarative object of any kind. Then there would be concrete subclasses for Param and for other libraries providing declarative objects. @philippjfr is the author of the Param class and would be the one to weigh in on how feasible such a task would be. As he's currently on holiday, I'm sure his answer would be that it's not feasible at all! :-)

As you mention, the other alternative would be to write a wrapper for a Pydantic class to appear as a Param class, and then use Panel's current param support as-is. I have no idea which option is easier, but the benefit of doing it at the Panel level is that it allows support for any Pydantic features not available in Param (e.g. a type that does not map directly to a Parameter type already available). If supporting Pydantic via mapping to Param, we'd first have to extend Param itself to accommodate that feature, then also extend Param to accommodate it, which seems trickier. But it's hard to say before trying it out which one would be more work, in the end.

Similarly, I'm also imagining Panel having support for traitlets, and in that case the default would be to map the the traitlet directly to an ipywidget, whereas mapping traitlets to Param first would lose special features in traitlets designed specifically for controlling ipywidgets.

[philippjfr]

Just chiming in quickly to say that I do think we should support pydantic (as long as it supports callbacks when settings params) and traitlets. Much less convinced that mapping traitlets to ipywidgets is a good idea by default. Also unclear what special features you're referring to there @jbednar.

[jbednar]

Traitlets has a couple of dozen types not available in param, such as Complex, ObjectName, and Set, and if we map a HasTraits object to a Parameterized object, we'd presumably need to add support in Param for those other types, then add such support to panel.Param as well. Not infeasible, but worth anticipating.

I'm also imagining that there may be ways that a traitlet can store values associated with it that go directly into the configuration of the corresponding ipywidget, but I haven't used traitlets enough to know if that is true. If it is, though, presumably mapping to ipywidgets would allow more customization than mapping to a Panel widget.

[philippjfr]

Oh right guess I'm misunderstanding, I'd be mapping from the traitlets to the widget, the indirection via a parameterized seems superfluous and complicated. If we do that mapping directly there is no benefit to using ipywidgets.

[bt-]

Hi All, nice work on the video! That was the first thing I spent time on when I started to dig into the new documentation.

I'm the kind of user @MarcSkovMadsen is describing in his second comment about the use of __call__ in the example. As someone from an engineering background and still earning best practices for writing classes, I can confirm it does make the example a bit harder to grok. I did learn something useful from reading the explanation in James' response. Maybe a little copying and pasting a sentence or two from James' response into a docstring for the call method and renaming the class would work? Something like:

class SumTwoNumbers(A):
    a = param.Integer(2, bounds=(0,10), doc="First addend")
    b = param.Integer(3, bounds=(0,10), doc="Second addend")

    def __call__(self):
    """Because this class does only one thing it's good practice to name that thing __call__, 
    instead of coming up with some other arbitrary method name for people to learn and keep track of. 
    I.e., __call__ just means "do the thing this class does - sum two numbers".
    """
        return self.title + ": " + str(self.a + self.b)