Establish live script example library repository

[vijayiyer05]

The MatNWB working group has been working towards establishing an organized library of example live scripts. The consensus view was that it would best be hosted by the DANDI archive organization (or an affiliated one).

Based on discussions, it is proposed to tackle this issue in stages:

• [x] 1. Stand up repository on a third-party organization, e.g. the INCF
• [x] 2. Document the repository organization guidelines, fine-tune if needed, and implement
• [x] 3. Upload initial examples (following the submitter guidelines per above)
• [ ] 4. Transfer repository to the DANDI archive organization (or an affiliated one)
• [ ] 5. Configure the DANDIHub MATLAB environment to pre-install the repository & set this as the initial working directory for new users.

See comments below and/or sub-issues for further notes regarding each step.

[vijayiyer05]

1. Stand up repository on a third-party organization, e.g. the INCF

Proposed name of repository is example-live-scripts.

This mirrors the example-notebooks repository currently established for Jupyter notebook examples.

[yarikoptic]

re name

• is there some semantic in using live making example-live-scripts being different from example-scripts in its nature?
  • but then it should also be open to non-matlab scripts submissions etc.
• may be they could be furnished as notebooks using e.g. https://www.mathworks.com/products/reference-architectures/jupyter.html and thus residing in the same example-notebooks?

[vijayiyer05]

2. Document the repository organization guidelines, fine-tune if needed, and implement

The MatNWB working group converged on the following guidelines for the example live script repository, which will be documented in somewhat more detail in the staged repository (e.g., in the README and/or wiki). 👉🏽These are general concepts & are freely shared as potentially also a model for other DANDI example libraries

Example Basics

• Each example artifact (original or derivative) is located in its own folder with its own license file
• Each example artifact (original or derivative) has a named owner (organization/lab/individual)

Example Submission Process

• Pull requests open to public to add new example artifact folders
• Repository admins will be reviewers to help ensure guidelines are followed

Example Folder Location

Folder location guidelines depend on whether example is hosted natively in library or external to library:

• Native to Library: <dandiset id>/<org/lab/individual- first commit year>/<mnemonic>/<code>
• External to Library: <dandiset id>/<external repository org>/<external repository name>/<code>

Native Example Guidelines

Natively hosted examples should use a permissive license option from ChooseALicense.com, e.g. MIT or BSD.

External Example Guidelines

Cloned Examples

IF the external example repository uses:

1. A permissive license option
2. GitHub releases

THEN the external example owner has the option to simply copy (clone) specifiic release contents into the DANDI example repository. This is (for some) a simpler option than the Mirrored Example option below (which uses git submodule).

This simplicity comes with one responsibility: the cloned example README should be minimally modified to include a header that 1) points to source repository & 2) identifies the release copied. If the copied release is updated (e.g., for new releases), this header should be updated accordingly.

Mirrored Examples

Besides the convenient case of Cloned Examples outlined above, git submodule is the expected workflow to mirror the example from the source repository to the example library.

Community Contributions to Examples

• Example owners can freely accept pull requests per usual.
• For native & cloned examples, pull requests for small, non-substantive changes/fixes can be submitted & accepted by administrators, without implying derivative work.
• More substantial changes would only be accepted by administrators as a derivative work contribution in a separate folder.

[vijayiyer05]

3. Upload initial examples (following the submitter guidelines per above) Initial Dandiset examples anticipated:
   • 000017 by CatalystNeuro
   • 000011 by Morteza Abbaszadeh, PhD
   • 000217 by Marielle Darwin, PhD

[vijayiyer05]

5. Configure the Dandihub MATLAB environment to pre-install the repository & set this as the initial working directory for new users.

MetaCell has begun work on refining the Dandihub MATLAB environment via Dockerfile edits. They will likely become the owners of this step.

[vijayiyer05]

re name

• is there some semantic in using live making example-live-scripts being different from example-scripts in its nature?

  • but then it should also be open to non-matlab scripts submissions etc.
• may be they could be furnished as notebooks using e.g. https://www.mathworks.com/products/reference-architectures/jupyter.html and thus residing in the same example-notebooks?

Thank you @yarikoptic for these good questions.

Live scripts are I would say more like notebooks (i.e., have graphics/rich text/equations/controls/etc) than ordinary scripts, so I think the 'live' qualifier is necessary.

Grouping these examples together with the (mostly or all) Jupyter notebooks in the example-notebooks would be the natural way to move in the direction I believe you're alluding towards. But this would have a couple of disadvantages in my view:

• The MatNWB working group has converged on a set of guidelines for organizing examples, and their derivatives and revisions. It's not intended to impose these on the broader set of notebooks, although they will be fully available to be replicated.

• Although there is growing interoperability between the MATLAB & Python ecosystems (including the MATLAB integration with the Jupyter desktop you linked to), there isn't notebook interoperability to my knowledge. So any library mixing both types of notebooks would just present as clutter for a viewer in either home environment.

[vijayiyer05]

• Task 2 is completed (with room for future improvement)
• Task 3 is completed (initial examples for Dandisets 000011 & 000207)