Ribbit-Network / ribbit-network-frog-hardware

The sensor for the world's largest crowdsourced network of open-source, low-cost, GHG Gas Detection Sensors.
https://www.ribbitnetwork.org/
MIT License
96 stars 26 forks source link

Frog server. The only container required to be installed on any hardware that'll become a frog. #139

Closed fosteman closed 2 years ago

fosteman commented 2 years ago

I think I might have come up with an architecture for the software. Just sketches right now. But I'm gonna go ahead and prototype it.

IMG_3648

This is the docker container we will install with a simple bash script on any compatible hardware.

Idea is to run 1 docker container on the hardware. Period. ... Or two. No, just one!

Node.js will do the job of running the show. It has scheduler (alike cron jobs), and can do a lot of damage in terms of running the frontend interface on the Frog (accessible on local wifi network) and interfacing with a db and our Ribbit network.

Ah yes, the db. We'll need a local (probably SQLite) db that'll persist sensor readings, authentication information, settings and other jazz that needs to be persisted with AND without the container. *Container will update with SF updates. DB needs to be run natively on the linux. The DB and other configuration files need to be stored in a persistent Docker Volume that does not get erased with eventual updates done to the container.

The core running on node should do the following:

  1. serve frontend on :80, allow regular folks to configure the Frog without ssh and with a user-friendly, touch-friendly interface accessible from their smartphones. And yes, the frontend. I'm thinking of a simple React app that creates the user story to empower any user to configure the frog with their two fingers.

Approximate user story is:

  1. Open the browser by a hostname. Somehow get the user to open the right ip address \ hostname on their local network. (need help on this one)

  2. Introduce them to the Frog in a couple of sentences, maybe host a youtube intro video.

  3. Authenticate. Let them create an account. This authentication will be managed by Firebase Auth - which will then create a unique API token for this user to use to write to the Ribbit Network.

  4. Configure frequency of sensor readings. Dark More \ Light mode.

  5. Visualize the process for the end user to understand wtf is happening while it's on. Perhaps do a test reading and show it in a Material UI Table, with a button "submit to Ribbit Network"

  6. Show a home dashboard with "Alive status", "Access to Data", "Settings" and finally "Logout". From this "Getting started" on this lovely React app, the settings are gonna be POSTed to the express.js node core.

The core will then persist all the auth, API keys, settings, in the local DB. Once that's done. We can get to work.

"Schedule" the jobs to:

  1. run python scripts to get sensor readings
  2. save in the local DB
  3. send to the network
    • Handle the errors and network loss.

https://www.loom.com/share/eb652854f4fa4613bc8bbab4e13096c4

fosteman commented 2 years ago

Firestore just stores the user profile info:

Screen Shot 2022-08-10 at 13 53 37
keenanjohnson commented 2 years ago

Hey @fosteman ! This is really interesting progress you are making! We haven't yet collected any identifying information about Frog Sensor owners, so figuring out if that's something we will want to do in the future is an important question that I don't want to consider lightly. I'm going to give this some thought.

Is the user info you are storing for the login stored on the cloud firebase or only on the Frog device itself?

What is the purpose of collecting it?

fosteman commented 2 years ago

Sure, @keenanjohnson , this is stored in the online DB. I believe owners should have an option to configure the frog offline without sharing names or whatnot. This online authorization flow could enable us to publish newsletters to emails of users, store usage analytics, etc.

fosteman commented 2 years ago

https://www.loom.com/share/eb652854f4fa4613bc8bbab4e13096c4

SambhaviPD commented 2 years ago

Hey @fosteman , this is cool progress. I'd one question though, the sensor readings are stored in local DB (firestore), and then sent to the network (the one that'll soon be replaced by InfluxDB)? So, there will be 2 copies of data, is that correct? Will the local firestore not outgrow depending on the frequency, or we thinking of any delete sensor data from local data > 'x' days type ?

fosteman commented 2 years ago

@SambhaviPD , not quite. Locally, there is SQLite database that will store the sensor readings. Configurable storage duration will be available from the setup sequence.

Firestore is just used for online accounts only - we’re still thinking how to use it best.

keenanjohnson commented 2 years ago

@fosteman this is excellent work so far! I really love the sequential flow view you have for onboarding someone and setting up a frog! That is vastly superior to what we have now!

image

Two Questions from me:

keenanjohnson commented 2 years ago

Closing this PR pending my questions above for now.