[API] Prototype API documentation in sidebar

[microbit-matt-hillsdon]

An initial prototype of API documentation in the sidebar that integrates with the editor.

This is early, exploratory work.

Aim to explore:

• Where we get the data from (stubs?, via Pyright?, precompute it for quick rendering?)
• Whether the stubs contain enough information to do a good job. Is manual curation required?
• What documentation improvements are necessary or desirable
• How we display it
• How we interact with the editor (e.g. drag and drop, other)
• Maybe, whether there's any interaction the other way, e.g. focus/goto docs as an alternative to docs on hover

[microbit-matt-hillsdon]

Status comment (I'll edit this as we go):

• Branch with API documentation document-style
• Branch with API documentation as an accordion (yes, I misspelled the branch 😞 )

Both pretty rough! We could focus on design / layout questions, documentation raising quality, drag-and-drop, progressive disclosure of doc string detail...

If we're generally happy with the direction, and don't seem likely to switch to manually curated docs for this API section, then there's work to be done to tidy up the Pyright side of this too.

Which modules?

I’ve left out modules for now that aren’t in our readthedocs docs: antigravity, builtins, love, this, errno, array, collections, gc, struct, sys.

From discussions with Giles it seems like we might want some docs for the builtins. The stubs currently don't have docs but they exist somewhere...

antigravity and love should probably remain undocumented easter eggs (you can autocomplete to them). The others are probably fine to add but perhaps we'd want to make some modules less prominent as we add more? Though I think a basic/advanced split is harder to justify.

Docstrings

I've truncated them at the moment but that's not really helpful. They're not suitable as-is.

Many don't contain a short summary that we could extract.

There is no systematic approach to parameter documentation.

They contain a lot of restructured text artifacts which we don't yet attempt to format or remove.

It'd be good to discuss what to do here.

Types

I've shown type information. I think it's useful in general but I'm definitely not the target audience. The very few places that use generics look arcane (e.g. random.choose).

Show inherited methods and/or details of the inheritance hierarchy. I think this just applies to the pins in our case.

More minor

• Dunder methods / operations (e.g. adding Images) aren't covered.
• Various issues raised against the stubs.

[jaustin]

@microbit-matt-hillsdon this looks excellent, thanks! I'm really surprised by how good it looks and how not overwhelming it is even in this early form. Very promising.

Which modules?

I think you've made a good first pass, and consider this ultimately an editorial choice. I think 'filesystem' probably wants to feature, a I think the accordion doesn't work as well for me because it makes exploration hard, but OTOH the long lists of things like melodies and images slightly confuse the accordionless one. I wonder if there's a version of accordions that is 'auto entering' once one is opened, so you can keep on scrolling as if it's one big list, or contract it? I( can't tell if in practice this will appear to be just the unexpanded list - I think it depends on when the collapse/re-expand happens, and whether you see the rest of the unexpanded items before )

Some simple things/responses to your questions:

• could we use 'returns' instead of '->': I think less notation is good, and I can stand the extra length of 'returns' I think. What do you think @microbit-giles
• ~As stated on Zoom, class inheritance isn't there, and I think it needs to be especially for buttons... https://microbit-micropython.readthedocs.io/en/v1.0.1/pin.html#classes~

Docstrings

I think, as you say, this needs discussion.

What you've generated looks good for giving us an overview and working out what needs to be done. I think you should probably just do as you suggest and create a good one-line summary for each item in the stubs. We can always refine these and it's good to have a starting point.

Type information

I think on balance this is confusing. Partly because when is shown there is not the actual text you get in the editor, but without types it's way more similar.

Places that are specifically confusing are things like this:

• string: str | float | int (💭 a string, that can also be a float or an int? )
• choice(seq: Sequence[_T@choice]) -> _T@choice (💭 whoops, look like the micro:bit Foundation's docs parser broke and there's some mistake in the rendering here)
• write(self: NeoPixel) Not sure this is really the fault of the typings, actually.

But, the legitimate followup is do we just show parameters only with their name? The ReadTheDocs docs are more verbose, and are name first: (also note the disparity between our autogenerated name for the first parameter and the name in the docs, string versus value)

I'd be keen to have some kind of ‘expansion’ where you can see the more detailed info, but kinda think maybe type info could go into the docs

[microbit-matt-hillsdon]

Discussed the latest apidocs and apidocs-accordian branches today and reviewed them alongside autocomplete and signature help (which are on main, but feature flagged).

There was considerable concern about the visual noise from autocomplete and signature help for learners.

We plan to:

• [x] Switch to a rectangular shape for the autocomplete drop-down and docs.
• [x] Show the same style of function signature as in the apidocs.
• [x] Show just the first line of documentation in autocomplete docs
• [x] Add a link to the entry in the Advanced sidebar.
• [x] For signature help, show just the friendly function signature with parameter highlighting.
• [x] We don't want to ship Advanced/apidocs without some more beginner focussed documentation. But we're happy to drop support for the apidocs branch and move the apidocs-accordian branch to a feature flag on main.

At some point we'll look at replacing the accordion with the similar design for the other documentation areas, but we only have wireframes at this point so we should hold off on that.

Other misc todos:

• [x] Inline multiline code blocks render poorly. Is there an incantation that'll make Pyright produce the right markdown? See e.g. ticks_diff.

Newer feedback that's not yet actionable:

• Maybe we want more in the signature docs after all. Perhaps an example? Perhaps docs for the active parameter? Or perhaps it's too much to add. Does having a simulator in future mean we'd need less here? Should users click through to the detailed docs?
  • Concluded that we'd try signature + active parameter doc + link to detailed doc.

[microbit-matt-hillsdon]

Did a little LSP digging and signature help doesn't contain the name of the function being called, which we need for our approach agreed above. It has a label, which has its type signature and some parameter information (also with types signatures). So this will definitely require changes on the Pyright end on a microbit branch of Pyright. Perhaps we could simply change the label. It seems like we should do the same for autocomplete documentation and share code if they already share code in Pyright.

To link to the "Advanced" docs entry, we'd want to know not just the name but the fully-qualified name. Perhaps we could just do that via Pyright? For autocomplete it's a little awkward as there isn't a discrete field to use. The type signature is currently just the first part of the markdown before the <hr>.

I'll have a first stab at these Pyright changes tomorrow morning. First of all though, I'll clean up the branch. Now that we can stop maintaining two branches we can tidy up without regard for making merging between them harder.

[microbit-matt-hillsdon]

I think I've got Pyright in a good place to support the changes above. It's now returning a simplified (no defaults) signature for the documentation (still in markdown, we'll have to live with an assumption that it can be parsed out of that).

The signature docs equivalent shows the defaulted parameters too as that might be your actual parameter position.

Both signatures use fully qualified names that we can use to unambiguously link them to our docs.

[microbit-matt-hillsdon]

Looking at active parameter docs, it seems we're not getting them for defaults. It does match the parameter but doesn't seem to return the docs.

This does work in a simple Pylance example so perhaps it's just a trivial error. I'll experiment some more.

The docs are just missing:

Seems likely this was broken by https://github.com/microbit-matt-hillsdon/pyright/commit/1e1a476845ed83eec18562cf9a2921b02a0f68e3 where I unintentionally changed what we pass into the extraction code.

Yep, easy fix:

[microbit-matt-hillsdon]

We've since aligned the Advanced / API documentation with the work-in-progress designs for the other (more beginner-oriented) documentation tabs. See PR #297 which can be previewed here. The micro:bit and Python toolkits are just mock content. The Advanced section is more complete, though we might reconsider the content to make better use of the space in the new design.

[microbit-matt-hillsdon]

Todos from PR that we'll cover off separately. I'm also going to close this initial issue but I'll make sure they all get a home elsewhere.

• [ ] Improve the code pop-up. Needs some serious time spent figuring out if it's the current approach with more care or there's another option.
  • https://github.com/microbit-foundation/python-editor-next/issues/301
• [ ] Add an entry with a drop-down, e.g. display.show with images.

More for me than the issue tracker:

• [ ] Iterate on aligning content drafts with the toolkit UI
• [ ] Catch up / chase actual designs

[microbit-matt-hillsdon]

Raised https://github.com/microbit-foundation/python-editor-next/issues/301 for code pop-up Raised https://github.com/microbit-foundation/python-editor-next/issues/302 for drop-down

Closing this initial prototype issue. Doubtless we'll raise more issues in future, e.g. to cover alignment with final designs and to address how we manage the content backing the micro:bit and Python toolkits, but this one isn't really actionable anymore.