Create Web GUI Dashboard with React (with NodeJS backend)

[vincentnam]

Project description

https://github.com/vincentnam/docker_datalake/projects/12

Web GUI : The service area is composed of different services that should improve the user experience. One of the services is the Graphical user interface (GUI) in a web browser. The reasons of the choice of web technlogies are :

• Easy to use for anyone
• Available on pretty much any device
• Great tools for GUI and data visualization

This GUI has several goals :

• Insert data into the datalake
• Download data from the datalake (from Openstack Swift or from any database / service in consumption area)
• Offer data visualization tools

The GUI has been designed as a 1 page dashboard with 3 tabs : insert, download and visualize tab. The differents components have to be designed and implemented as a card (React / bootstrap concept : https://react-bootstrap.github.io/components/cards/) to make it possible to easily modify, compose and adapt tabs.

Languages : Javascripts, HTML / CSS Tools : react, nodeJS, d3.js, bootstrap

The dashboard should be a multi-tab page with user authentication based on Datalake central authentication tool (Openstack Keystone). The other tab should be available until the user is authenticated.

• [x] Initial dashboard
• [ ] User tab for authentication

The insertion tab has to be easy to use to make it possible for people that doesn't know computing science to use the datalake. It has to integrate a drag'n'drop dropzone for files input. The data has to be qualified : technicals metadata should be inferred as much as possible but for metadata that couldn't or for semantical metadata, it should be possible to easily add it. As the project is in a great part a research project, it is linked to semantic web and ontologies and this insertion tab should integrate a way to easily insert this kind of data ( drag'n'drop tool for tags / label choice, classic form (dropdown menu)) that would generate a JSON document. The metadata should make it able to :

• Insert a list of mandatory metadata
• Chose among a list of metadata new metadata (from pre-defined ontologies)

Data insertion tab :

• [ ] Dropzone creation

Metadata form insertion :

• [ ] Tag / label drag'n'drop selection

• [ ] Dropdown menu

• [ ] SLO implementation (Openstack Swift insertion function : Static Large Object)

The data visualization tab goal is to :

• Quickly see if data have been inserted in the good container
• See what data are already in the datalake for a user
• Provide complex data visualization for crossed data (as the visualization of time-series for sensors on a map with their location) Tools should be customable by setting differents sources, drag'n'drop card for tools, anything is possble here until it is usefull for anyone. In the end, it should make it possible to create dashboard, save configuration for further utilization. It could integrate other visualization tool (as InfluxDB or neo4J web interface).

Data visualization :

• Basic visualization :
  • [x] Graph visualization
  • [x] Time series visualization
  • [ ] MongoDB collections content visualization

Maybe not useful :

• Complex visualization :
  • [ ] Map with time series
  • [ ] ... ?

The data download page should make it possible with graphical tools to select a data / a dataset and download it on local host.

Data download page :

• [ ] Visualization of download source list
• [ ] Download basket component
• [ ] Download button
• [ ] Visualization of differents data and selection tool

The project will be splitted in several big stages.

[vincentnam]

First stage :

The first stage goal is to create the base app for the GUI.

App architecture :

• [x] Create the initial dashboard with several tab
• [x] Create 3 tabs : insert, visualization and download tab
• [ ] Make a design for the application
• [ ] Create a log-in page

Insertion tab :

• [ ] Create first dropzone for file inputs #10

Visualization tab :

Basic visualization

• [x] Basic graph diagram tool
  • [x] Initial features
  • [ ] Rework and upgrades (design and features)
• [x] Basic time serie diagram tool
  • [x] Initial features
  • [ ] Rework and upgrades (design and features)

Download tool :

• [ ] Graphical tool to download a file / data
• [ ] Visualization tool for data that are available to download for an user

[vincentnam]

Second stage :

The second stage goal is to upgrade features, design for the web GUI. One important feature is the implementation of drag'n'drop as much as possible.

App architecture :

• [ ] Create a way to create custom dashboard (save as an user configuration)
• [ ] Create a drag'n'drop tool for the construction of the dashboard : Select a list of tool among the whole list of tools developped in the web app to create a dashboard and save the configuration for a later use

Insertion tab

• [ ] Visualization tool to see if data has been well inserted in the datalake
• [ ] Visualization tool of metadata to see what data has already been inserted

  Visualization tab

  Complex visualization :

• [ ] Map with insertion of time series plot for each point on the map (exemple : meteo station)
• [ ] Heat map

Download tab

• [ ] List of data that are available as a draggable list of items
• [ ] Drag'n'drop cart for downloading data

[vincentnam]

Third stage :

The third stage goal is to finalize the application and develop new features.

App architecture :

• [ ] Design a GUI integrator tool to integrate easily other tool already developped
  • [ ] Integrate in the app InfluxDB visualization tool (in a card and / or in a tab)
  • [ ] Integrate in the app Neo4J visualization tool (in a card and / or in a tab)

Insertion tab

• [ ] Stream creation tool (through Kafka)
• [ ] .. ?

  Visualization tab

• [ ] Integrate new visualizations tools :
  • pretty XML visualization
  • pretty table visualization
  • ... ?
• [ ] .. ?

Download tab

[vincentnam]

Idea :

• Visualization of representative data over a dataset