toni-neurosc
commented
2 hours ago Hi @timonmerk and @elizaveta-terek, since I am the one who proposed building the GUI as a web-app, I would like to give here a brief explanation of the tech-stack we will be using and the reason for each of the choices:
Frontend (React SPA)
-
Javascript runtime + package manager: Developing web apps that use Javascript as a programming language typically requires not only running code on the browser (which has a JS engine build-in), but also running development tools locally on the computer, which also need to be installed from a repository along with their dependencies. The typical runtime that allows running JS apps locally is Node.js, and the standard repository for JS packages is NPM (Node Package Manager). Therefore, typically one would install Node.js and NPM by using a tool called NVM (Node Version Manager). As you can see, this is already getting fairly complex, for that reason I usually prefer using Bun (https://bun.sh/) which is an all-in-one JS runtime and package manager written in Zig that is super-fast and has a syntax almost identical to npm/node. But the choice on this tool does not really matter as it does not affect the rest of the workflow.
-
Development environment: A typical local JS tool is a development environment that takes care of a series of things:
- Create the scaffolding required for the app (folder structure + boilerplate code)
- Running a development server that builds the app real-time so you can see the changes as you change your code
- Compiling and packing all the code into "static" files that are then served by the web-server.
The most popular dev-env for JS is Vite. It is a combination of several tools: a project setup tool, a dev server, a plugin manager, a Typescript compiler (but it supports plain JS too) and a webpacking tool to build and deploy the app. You can install and run Vite using any JS package manager: https://vitejs.dev/guide/#scaffolding-your-first-vite-project
-
UI framework: When developing an app in JS, there are a couple things that you want to do that are a pain to do with plain HTML/JS/CSS. This would include:
- Creating custom UI components with encapsulated behavior and style that can be re-used in different places, like for example creating a button that you can put in your HTML by writing
. - Passing JS variables into the HTML and CSS code.
- Maintaining client-side state: meaning that the state of the UI is maintained client-side and updated as the user interacts with the app without the need to pass data to the server or request new HTML from the server. Of course, we can still pass data to the server when we need to.
- Creating reactive UI's: this means basically that the UI updates itself as the data changes without us needing to do anything except handle the underlying state in the JS code.
We have chosen React as a framework for 2 reasons mainly:
- It's the most used UI framework, so as @timonmerk mentioned, there is a lot of support for it and teaching resources, as well as ready-made, open-sourced UI components that we can copy or adapt to do things quickly.
- Plotly maintains a React component library that we can immediately start using to display plots: https://github.com/plotly/react-plotly.js
React is, nonetheless, not without some disadvantages:
- It's a batteries not incluced kind of framework, which means that there are some things that it can't do out of the box (e.g. client-side routing, database-connection, etc.) so it relies on other open-source tools to do those things such as React Router.
- It's one of the oldest frameworks and has accumulated a lot of complexity with the years, so the learning-curve is a bit harder than other frameworks.
- It's not the most performant, so we will have to see how it handles the plotting of real-time data to a reasonable framerate.
- Creating custom UI components with encapsulated behavior and style that can be re-used in different places, like for example creating a button that you can put in your HTML by writing
-
Client-side routing: This is a bit more technical, and it has to do with the concept of SPA or Single-Page Application. The conept of "routing" will become clearer when I explain the backend, but it basically means that the app can move between different sections of the app (e.g. pynm.app/open-stream and pynm.app/view-stream) without reloading the interface, replacing only the necessary parts of the interface at each time.
Backend (Python server)
Now let's talk a bit of backend, which will be handled entirely in Python:
-
Web app framework: Usually a web-site has different pages inside it, for example, and https://en.wikipedia.org/ and https://en.wikipedia.org/wiki/JavaScript are 2 different URLs that contain different HTML code. In the context of a web-application, sometimes this URLs are not just folders with HTML files inside, but rather are API-endpoints that return data that the frontend requests to the server. For example, if we wanted to get data from our server to show it in a graph, we could create an API-endpoint called "http://myapp.com/data", and when we make an HTTP request to this URL, instead of returning HTML it will return data, typically in the form of a serialized data format such as JSON. From the perspective of the web-app server, these URLs together with the associated response are called "server routes" and the functionality is called "routing".
So this is more or less what the server does in the context of a web-app and for doing this we need a Python package that handles everything necessary to make this happen, including routing. Usually we call this a backend framework and Python has different options, with Django being the most popular but also being fairly big and complex and requiring a lot of boilerplate. Instead, at least for now, we will be using Flask, which is a very minimal framework that works with very few lines of code.
-
Web-sockets: communication between backend and frontend is typically done by using API-endpoint routes that return serialized data, but in the event that we need an event-based method that allows real-time bi-directional communication between frontend and backend that also supports sending binary-data, we can use the Websockets protocol. I'm still new to this as well so I'm learning how to use it, but I think this is the best way to share the raw data and calculated feature data stream between the backend and the frontend for graphing purposes. To avoid using the low-level Websockets protocol directly, a typical solution is to use a library called SocketIO that has Python support and particularly a Flask integration package called flask-socketio that we will be using
Overall Goal:
Milestones:
Ground work (basic):
Refine (intermediate):
Perfectionise (PRO):
Useful tools: