Restructure, moving list of approved widgets to external repo

[AlanSl]

We've re-examined the repo structure and are proposing this small change that would ease both third-party contributions and ongoing core development and maintenance. In addition to the changes here, a new repo libp2p-introspection-widget-list will be created that houses the list of approved widgets* to be displayed in the catalogue, and which 3rd parties interact with when contributing their own work.

That new repo will provide the hub for community activity around creating, requesting and discussing ideas for widgets. This will allow this monorepo to be tightly focussed on the core project itself. The list repo will contain very little code beyond providing the list of approved widgets that the official catalogue consumes, which will preserve the benefits of the monorepo design such as reducing development time and easing core project contributions.

*We're also suggesting "widget" as the standard term for one published visualisation/interface that can be shown and selected in the catalogue. This avoids ambiguity where one "widget" may contain multiple "components" and "visualisations" such as side-by-side linked charts.

Proposed workflows:

For a third-party developer who is developing a widget

1. In an empty repo, npm init, npm i @libp2p-introspection-ui/sdk
2. I'm looking into a simple generate script in the sdk, like npx generate my-widget-name, which installs dependencies such as react and styled-components, creates the main files and the exports expected by the catalogue, wires in storybook with standard settings and sample data, adds default lint settings, etc. This is a pattern widely used in React projects and is a good way to lower the barrier to entry and boost developer confidence.
3. They work on this, following the documentation linked to from https:// www.npmjs.com/package/@libp2p-introspection-ui/sdk
4. They publish to NPM on their own account, and run a deploy script provided by the SDK that hosts a demo of the widget with its own shell (timeline, file upload button etc) on the user's own Github Pages
5. After checking the contribution guidelines on https:// github.com/libp2p/libp2p-introspection-widget-list, they submit a 2-line PR to that repo adding their widget as a dependency and including it in the exported list, following a standard format including a link to their demo. This PR is given tags depending on whether they consider the widget to be complete and worthy of formal review leading to inclusion in the official catalogue, or, whether they consider it work-in-progress and want more informal input and chatter from the community.
6. Users will be able to view the Pull Requests to the repo, and depending on the tags chosen they can filter and find the community contributed non-vetted/merged widgets and try out those deployed widgets. Because the users will be finding and interacting with these widgets through the PR workflow of GitHub, it will be apparent to the users that these are not maintained by the core team and therefore should not be trusted as such. This would facilitate spontaneous community activity and chatter, without exposing security risks on PL-branded pages.
7. If the contribution is successfully vetted, the PR is approved, and that specific version of the widget is added to the list imported by the main page of the catalogue, and displayed for users to use without warnings.

The activity around the https:// github.com/libp2p/libp2p-introspection-widget-list will consist of community members and PL staff discussing 3rd party contributions, sharing advice, posting issues with ideas for widgets they would like to see or projects they would like help with, etc etc. Such activity building on the project is then separated cleanly from activity around maintaining and developing the underlying project itself.

A third-party widget contributor's only interactions with the main repo here (which will be https:// github.com/libp2p/libp2p-introspection-ui/) would normally be following documentation links from the NPM page for the @libp2p-introspection-ui/sdk, looking at core widgets as examples to follow, or posting bug reports or feature requests if they have any.

While the initial discovery method of non-vetted widgets is expected to take place using GitHub PRs, it could be decided that later work will add these widgets to a "pending approval" list that can be viewed from a secondary page in the catalogue. These listings would link to the build of a widget on GitHub Pages corresponding to the contributor's own account, with warnings before following the link that this hasn't yet been vetted and users should therefore take care around allowing access to local websockets etc.

Keeping as much community activity and chatter as possible handled through GitHub and contained in this repo will be a good way to keep a low barrier to entry and should be very compatible with the GitHub-oriented workflows in other LibP2P projects.

For maintainers and developers working on the core product

All maintenance and core dev work will be on https:// github.com/libp2p/libp2p-introspection-ui/ and will be more project focussed. The SDK, catalogue app shell, JavaScript protobuf processing, and built-in officially maintained widgets will be independent Lerna packages within this monorepo, which means they have their own versioning and are independent packages on NPM.

This brings the benefit that development and maintenance is eased by being able to update multiple packages in one PR. For example, any changes to the protobuf data will likely require changes and new releases for the catalogue, SDK, and all core widgets; but it is also possible for a PR to update only one core widget, or to migrate a reusable feature from a core widget to the SDK, with only those packages having new releases and updated version numbers.

It is also a very common pattern widely used in the React ecosystem and will be familiar to third-party developers most likely to want to contribute either to the main project or by creating their own widgets.

[AlanSl]

Tests are failing because https:// github.com/nearform/libp2p-introspection-widget-list doesn't exist yet 🙂 will add once we've got an in-principle go-ahead for this approach

[AlanSl]

We've talked about this some more in light of some more detail added to the contribution model, and (including Raul's naming suggestions) latest thinking, it would look like this:

• libp2p-observer - monorepo that publishes the following independent packages:

  • @libp2p-observer/sdk - the React Hooks-based UI binding framework for widgets
  • @libp2p-observer/proto - protobuf data handling and sample data
  • @libp2p-observer/catalogue - base components for users creating their own catalogues and for the official "Observation Deck"
  • @libp2p-observer/create-widget - generate script for quick start of new widgets with one-line setup, getting them ready to develop, use data, and build. Can be run from NPX without installing or with npm init @libp2p-observer/widget or yarn create @libp2p-observer/widget.
  • The official, built-in widgets: @libp2p-observer/connections-table, plus streams, DHT, network
  • Hosts a build of @libp2p-observer/catalogue on this repo's github pages, pulling in the official built-in widgets only, rebuilt on each push to master, and is used for staging

• libp2p-observation-deck - community hub for users and developers of libp2p-observer widgets

  • Exports and publishes the list of officially-vetted third-party widgets, and houses PRs and issues where the community share and discuss new and potential widgets
  • Hosts the build of the official "LibP2P Observation Deck", pulling in @libp2p-observer/catalogue and applying both the official built-in widgets and approved third-party widgets. Updates on each push to master (i.e. each addition, removal, or version number bump of a 3rd party widget), and rebuilds from any new releases of @libp2p-observer/catalogue on a timed cycle (e.g. weekly or nightly)

This is mostly the same as the above plan, keeping all the benefits in terms of productivity plus clear separation of user/contributor and project/maintainer activity, but with a clearer live staging environment and clearer support for users putting together their own dashboards/catalogues, and better naming.

[AlanSl]

Plan has changed a little making this obsolete