[Discuss] Developer docs website

[ashwin-pc]

Is your feature request related to a problem? Please describe.

This has been brought up a few times in different places. Opening the issue here since it looks like the goal of the documentation and project website do not cover developer docs. There are 2 personas that i'm trying to solve for here.

1. A plugin developer: These are users that ideally should not need to look at the OSD codebase to make plugins that extend OSD. This is mostly possible today with the current plugin architecture, but the documentation needed to help such a developer is incomplete and scattered across the repo. This makes it such that a plugin developer must also be very familiar with OSD code and adding unnecessary friction to developing plugins.
2. OSD Developers: These are users who contribute directly OSD. They too require some of the information that plugin developers need, but they should also be familiar with OSD code and can rely to some extent on readme's in the code.

While markdown is powerful, there are some pitfalls by only relying on it to address our gaps in documentation for the two persona's mentioned above:

• The docs are scattered (they are kept local to the component they are describing) so finding the relevant information can become tedious.
• Cant incorporate features such as tutorials, search and nav tree to name a few
• Requires the user to explore the OSD repo (For plugin developers, this is not ideal)

Describe the solution you'd like

A developer docs website that contains informations such as:

• Overview
  • Core concepts
  • High level architecture
• Plugins
  • Plugin Architecture
  • Plugin framework
  • Core plugins and docs for each plugin
• Tutorials
  • How to build a plugin
  • How to add a visualization
  • How to create a new expression
• Best practices
  • Performance
  • Styleguide
  • Testing
• API's
• Other resources
  • Local env setup (this is the same for contributors and plugin developers today but that might change in the future)
  • Design system (EUI or its equivalent that we will use)

Describe alternatives you've considered

This could also be an auto generated doc site using the readme's in the repository. However this will be quite limiting for the breath of information that we have. Once we decople plugin development from having to build Dashboards, plugin developers should not have to refer to docs from the repo directly. A separate developer doc site solves that problem too.

Additional context

Existing issues for context:

• https://github.com/opensearch-project/documentation-website/issues/492
• https://github.com/opensearch-project/documentation-website/issues/419#issuecomment-1048102288
• https://github.com/opensearch-project/OpenSearch-Dashboards/issues/778
• https://github.com/opensearch-project/OpenSearch-Dashboards/issues/1100

[CPTNB]

I think you have to start with something that is index-able and externally link-able. Plugin devs will want to google APIs or link to APIs on stack overflow, and contributors will want to also browse docs.

So that means a website with:

• anchors to sections / APIs /pages
• probably a blog section for longer form brain dumps about philosophy
• probably linking between pages
• basic SEO fare like sitemap and robots (unless I'm a dinosaur and Google doesn't use these anymore)

You could start with a tool that picks up existing docs and assembles them into a graph using some best-guess heuristics like "package.json depends on", then displays the markdown as a web page. You could also just hand-create the graph links.

Personally, at least in the early stages, I don't think this should be anointed with any authority by putting them inside the broader official docs site.