search

FlyBrainLab / NeuroMynerva

NeuroMynerva integrates the Fruit Fly Brain Observatory (FFBO) Project with JupyterLab, providing CAD capabilities and a neuroscientific data exploration platform.
BSD 3-Clause "New" or "Revised" License
14 stars 5 forks source link

Handling Server Settings & Multiple Datasets #20

Closed TK-21st closed 4 years ago

TK-21st commented 4 years ago

NeuroMynerva V2 aims at supporting connection to multiple servers and switching between datasets on a given server.

Nomenclature

We need to standardize nomenclature

Use Case 1

Use case 1 is focused on browser entry point, where user experience mimicks logging on to a data server with options of which available sessions to connect to.

  1. After launching JupyterLab, the user should be able to click on a button to view currently available server backends, with metadata showing information such as ip address, and a list of available datasets
  2. Ideally, there should be a way to test to see if a connection can/cannot be established with one of the servers at any given time.
  3. In addition to making edits in a popup dialog, user should able to add/remove available dataset settings.
  4. User should be able to easily change settings of the currently available server settings, primarily with regards to secrets, url, ssl, etc.
  5. User should be able to export/import settings to/from JSON or INI file formats.
mkturkcan commented 4 years ago

Nomenclature

Crossbar Server: FFBO Processor NeuroArch Database: dataset

Suggestions for Data Structures

  1. Available "species" and their configs can be stored using the following dictionary structure: {'Drosophila melanogaster (default)': { 'config': 'drosophila_config.ini', 'datasets': {'larva': {'3d_settings': 'https://cdn.rawgit.com/fruitflybrain/ffbo.lib/master/larva_init.json', ...}, 'hemibrain': {'3d_settings': 'https://cdn.rawgit.com/fruitflybrain/ffbo.lib/master/hemibrain_init.json', ...}}, ... }, ... } where keys are names of the species to be seen in the frontend, and values correspond to the config settings and dataset-specific visualization settings. 'config' can be a .ini file, or a dictionary-fied version of the .ini file. 3d_settings can be a json structure, a link to a json structure hosted online (for example on GitHub), or a link to a local json file, detailing the meshes to be loaded. Over time, I expect us to add more keys, so the parser and user interface for editing this data structure should be flexible.
  2. Users can edit this config in frontend, retrieve the frontend config data in the backend, or send an edited config from backend to frontend.

    Use Case (Custom Model)

    In this use case, a user designs a computational model with custom server and visualization settings, served as a number of .py files and a .ipynb notebook. The user should be able to load their configuration without having to require users to take a number of steps in the frontend:

  3. An init function updates available species in the frontend upon execution of the first cell in the notebook.
  4. Further, the function automatically switches the species and dataset for frontend components associated with the current kernel. The new species is now available as a setting when creating a workspace for new users.
TK-21st commented 4 years ago

Current implementation uses processor and the corresponding schema to handle backend settings.

For suggestions regarding additional information related to the content of the schema (e.g. 3d_settings) please open a separate issue.