How should we create documentation of the operations, modules, datatypes a plugin provides?

[caro401]

This documentation existing is really important for end-users as it tells them what each plugin can do for them. The content of this documentation is contained in the python code for each plugin, and can be accessed via the kiara API.

Existing examples of this documentation are here, and the content comes directly from the module code eg this datatype docstring

There's a distinction between plugins created by the internal team, and those created by external people. We probably want documentation for internal plugins to live in one place, alongside the main docs, assuming this matches with @MariellaCC's survey results.

But we also need external plugins to be able to create standalone docs, as they currently do. The plugin template should come with a github action that deals with building and publishing these docs automatically, with no intervention from the developer.

Because we can get all the content via Kiara's API and the docs can be built in Github actions, we aren't constrained to just using a python-based solution for this. It may be advantageous to use a javascript-based solution, using at least some of the technologies we use for the mini-apps to enable component and styling reuse. This would probably be Svelte for components, and possibly SvelteKit for site building.

There's some vague prototypes of what this could look like in this repo which could have some code that could be adapted into components, or a static site builder, but it isn't currently good, tested or documented.

Open questions I can think of

• what features does a standalone plugin docs site need to have? Is just one big blob of HTML enough or does it need things like a search box, navigation etc.
• What technology should we use to make the site(s)?
  • How easy is it to integrate components from the mini-apps into that (via an npm package?)?
  • How easy is it to make something that can build completely standalone with just input from a cookiecutter template?
  • what are other members of the team comfortable maintaining? I'm most comfortable using svelte/kit and/or astro, but maybe others have other areas of expertise. @makkus does this match with your comfort zone?
• do the docs need to be able to cover multiple versions of a plugin? are there nice ways to communicate if a thing was added/removed in a particular version of a plugin?
• do these docs need to support different languages? I guess probably not, since the content coming from kiara itself is (currently?) only in English?

[makkus]

Agree with everything said here, thanks for the write-up!

what features does a standalone plugin docs site need to have? Is just one big blob of HTML enough or does it need things like a search box, navigation etc.

A minimal navigation would probably good, with an overview page of what components are available in a plugin and a short description, and links to more detailed information.

One thing that would be really good to have is every component having it's own predictable url (or url & anchor), something like https://dharpa.org/kiara/plugins/PLUGIN_NAME/operations#table.sort (just an example, can be anything that works best -- if we can get that to work for external plugins it would be nice, but I'm mostly concerned about our own official plugins here), That would make it easier to integrate links into our app(s) to send users to places where they can get more information if they want/need.

What technology should we use to make the site(s)?

I'd also lean towards svelte/svelte-kit. It's easy to pick up, very powerful, actively developed, and has a high user satisfaction.

streamlit is nice in that it is Python only, but my experience working with it makes me think it'll be too inflexible for our needs, since a lot of the stuff we are doing UI wise is not a trivial top-to-bottom app. @MariellaCC has experience with Plotly Dash, so she could comment on that. My instinct is going for a full JS framework though, because in the end it just gives us the most flexibility and powerful features without having to worry about yet another intermediate layer. For extra cookie points, it would be interesting if we can make it so that we create ipython widgets/streamlit components/other tech components by wrapping our svelte components in an automated way. I do see some interesting use-cases for something like streamlit (simpler apps, simpler deployment, prototyping, quick 'UX design sketches', ...). We probably can't afford to spend too much time on this, but if we could at least in theory prove that it would be possible (which I think it is), it would be a valueable thing to have IMHO. But, as I said, this should not keep us from the more important things, and can always be done later on.

do the docs need to be able to cover multiple versions of a plugin? are there nice ways to communicate if a thing was added/removed in a particular version of a plugin?

Given our focus on reproducability/lineage/metadata, I do think it would be good to keep the history. We haven't really talked about how we version plugins, but keeping one version of documentation per git tag would be good. Sidenote: I haven't really thought through all the implications, but in theory data types and modules would have independent versions from the plugin that contains them. For example, a module doesn't need to change because the plugin it contains is updated (e.g. by adding more modules). It's a complicated issue though, with lots of things to consider, so I just put this here in case we stumble upon something in our implementation where this might be relevant.

are there nice ways to communicate if a thing was added/removed in a particular version of a plugin?

Interesting question, didn't cross my mind yet. We could maybe maintain a database with all modules/data types versions and their metadata by feeding into it at every release of a plugin? That way, we could present that information in any way that becomes useful? A database like that might also prove useful for other metadata related features later on, for example we wouldn't need to have a running kiara service for some type of information rendering... Hmmmmmmmmm, something to mull over....

do these docs need to support different languages? I guess probably not, since the content coming from kiara itself is (currently?) only in English?

I guess, depending on how much effort it is, maybe we could use the i8n features that come with our selected framework, but only implement English for now? But that's mainly a trade-off between how much resources we can afford to spend on this now, against how likely it is in the future that we'll need it and how much more difficult it would be to implement it without this sort of preparation. I guess a question to answer once we have a rough plan and priorities? It would be best practice to prepare for it I guess.

Speaking about best practices, what should/can we do about accessibility? I don't have much experience in that field, but my guess is it's much easier to not have to retro-fit that, but consider it from the beginning.

[caro401]

One thing that would be really good to have is every component having it's own predictable url (or url & anchor),

Great idea! like a permalink for a component. Would/could this also include version information? Could the plugin package be able to communicate what the base URL for its docs is, for non-core plugins that can't use dharpa.org/kiara, so we can always give full links?

My instinct is going for a full JS framework though, because in the end it just gives us the most flexibility and powerful features without having to worry about yet another intermediate layer.

My one concern with JS is testing - not applicable so much to these docs, but definitely to other kinds, how do we make sure the code contained in the docs runs and is correct. IIRC there's already something that does this for the existing mkdocs implementation, but I'm not sure how I'd do that via JS

but keeping one version of documentation per git tag would be good.

this would be a kind of deploy once and leave running without ever changing thing? Does a git tag correspond to a python package version that can be installed? How do we know what version(s) of kiara core it could be installed alongside to get the documentation from the plugin via kiara's API?

It would be good to document how we do versioning, and what versions (of packages and of other things?) mean for end users, so they can know whether they can safely update things they are depending on. I'd kind of assumed it was something like semver at the python package level, but that's not the case? I'll add a note to the 'concepts' bit of the core docs outline to remind me this needs doing - 5aaba0b

Re permalinks for things, would it then look like /plugins/PLUGIN_NAME/vX.Y.Z/operations/table.sort or is it more meaningful to version the operations etc themselves rather than the plugin? /plugins/PLUGIN_NAME/operations/table.sort/vX.Y.Z

are there nice ways to communicate if a thing was added/removed in a particular version of a plugin?

A database is a nice idea but not something I'd feel confident maintaining. Would something like a json/yaml file in the docs github repo be able to achieve something similar without needing to maintain a database server?

We could consider publishing changelogs for each package alongside this documentation, so users can get summaries of what changed in each version? Requires a bunch of effort to write for the plugin authors, but I think would be really useful for end users. (I keep getting tripped up by changes in the names of operations in the network analysis plugin, being able to see a list of what changed between versions and maybe something like a migration guide would be super useful)

maybe we could use the i8n features that come with our selected framework, but only implement English for now?

sounds good. It would be nice to make it easy to do translations in the future, but this isn't something I have any expertise in (technically or linguistically! my other languages are not fluent enough to write technical docs in).

Speaking about best practices, what should/can we do about accessibility? I don't have much experience in that field, but my guess is it's much easier to not have to retro-fit that, but consider it from the beginning.

Sorry I forget accessibility is even a question! I'll make sure that's done to the best of my ability from the start - I know a decent amount and care a lot. For our purposes, it will mostly look like generating semantic HTML, making sure everything is possible and logical to navigate with a keyboard, and using a clear colour scheme. Might also be worth adding an accessibility statement, to the effect of we're doing our best as a small team, tell us if something isn't working for your needs?

[caro401]

Proposal for the technical side of how to make a start on this in #3, but I'll leave this issue open since there's still a bunch of open questions.

Having done some research, it seems that there isn't a particularly good solution for translation in sveltekit at the moment. Since kiara's internal docs are also only in English, and the team has very limited resources, I think we should not consider translations for now, and wait until there's a need from kiara's users, more dev capacity and hopefully the sveltekit community has settled on a decent approach to take.

[makkus]

Great idea! like a permalink for a component. Would/could this also include version information?

Yeah, I think that would be best. Depending on how we version the docs themselves, it might just be a matter of <base_url>/<plugin_version>/<rest of docs> ?

Could the plugin package be able to communicate what the base URL for its docs is, for non-core plugins that can't use dharpa.org/kiara, so we can always give full links?

Yes, currently I've hardcoded dharpa.org in the template I think, but that would be easy enough to change and add to the metadata of the project.

this would be a kind of deploy once and leave running without ever changing thing? Does a git tag correspond to a python package version that can be installed? How do we know what version(s) of kiara core it could be installed alongside to get the documentation from the plugin via kiara's API?

The only piece of information we have is the kiara pip dependency (which can be investigated at runtime -- or in a pre-render step). I think we have no choice but to assume that this is correct.

Re permalinks for things, would it then look like /plugins/PLUGIN_NAME/vX.Y.Z/operations/table.sort or is it more meaningful to version the operations etc themselves rather than the plugin? /plugins/PLUGIN_NAME/operations/table.sort/vX.Y.Z

I guess that depends on the framework we are using in part, and whether we link to our 'central' site, or each plugins documentation directly for the permalinks. The latter would have the advantage of automatically also working for external plugins. But would mean that we would not link to our 'central' official doc, but each plugin independently. Maybe if we go with the first option it would work for both cases?

[makkus]

A database is a nice idea but not something I'd feel confident maintaining. Would something like a json/yaml file in the docs github repo be able to achieve something similar without needing to maintain a database server?

Yeah, as we talked about earlier, I think that is a great idea, and much better.

We could consider publishing changelogs for each package alongside this documentation, so users can get summaries of what changed in each version? Requires a bunch of effort to write for the plugin authors, but I think would be really useful for end users. (I keep getting tripped up by changes in the names of operations in the network analysis plugin, being able to see a list of what changed between versions and maybe something like a migration guide would be super useful)

Yes, I think we can partially automate that by establishing best practices for git commits, and then automatically generating those changelogs. It's going to be messy and incomplete either way though, I fear.

As for changes in operation names, this is something that happens at the moment because I don't have enough experience yet to have a sense of how to design modules well from the beginning, but I really want to minimize/discourage/forbid that sort of change, if at all possible. Modules should have a stable interface, and if a change is needed, the old module should stay in place and a new one with a different name created. Maybe we can be a bit smarter and have those module sub-versions and do some basic dependency resolution, but that goes back to our problem with having enough time and resources.

[MariellaCC]

streamlit is nice in that it is Python only, but my experience working with it makes me think it'll be too inflexible for our needs, since a lot of the stuff we are doing UI wise is not a trivial top-to-bottom app. @MariellaCC has experience with Plotly Dash, so she could comment on that.

@caro401 I agree that Streamlit comes with too many limitations as soon as the scope of the app is beyond a certain level of complexity (it is difficult to give a precise scale). Still, I found it helpful for prototyping purposes or to create quick dashboards up to a certain level of complexity. For my needs in the past, Streamlit was not flexible enough, so I experimented with Plotly Dash. Before that, I had experimented with Flask combined with React (which remains my preferred experience in terms of flexibility but the choice of tech was quite arbitrary). As I am not specialised in front-end development, it is difficult to have an opinion on a suitable solution. My comment is only about the flexibility needed from past experiments, and I agree that, in general, Streamlit is too inflexible for sustainable mini-apps.