merge Browse Sites and Time Series Analyst tabs?

ODM2 / ODM2DataSharingPortal

A Python-Django web application enabling users to upload, share, and display data from their environmental monitoring sites via the app's ODM2 database. Data can either be automatically streamed from Internet of Things (IoT) devices, manually uploaded via CSV files, or manually entered into forms.

BSD 3-Clause "New" or "Revised" License

31 stars 8 forks source link

merge Browse Sites and Time Series Analyst tabs? #463

Closed dbressler75 closed 3 years ago

dbressler75 commented 4 years ago

For clarity it might be beneficial to merge the Browse Sites and TSA tabs (in upper left of main MonitorMW page) into one main tab for accessing and graphing data. Currently the two tabs seem to function slightly different from one another but do not provide dramatically different functions so it just confuses the process.

SRGDamia1 commented 4 years ago

You mean here? They do provide dramatically different functions, they just happen to both open up a map view.

SRGDamia1 commented 4 years ago

Any time you click on something "tsa" you can think "I'm moving over to a totally different website that just happens to have most of the same data on it."

dbressler75 commented 4 years ago

But what are the dramatically different functions? - once you're into TSA in Browse Sites it's the same TSA. The main Time Series Analyst just tab on the home screen just takes you to that map with nested maroon colored sites.

SRGDamia1 commented 4 years ago

yeah.. I guess they do kind-of feel the same. But TSA is still basically a totally different website that just happens to share data.

aufdenkampe commented 3 years ago

@dbressler75 and @SRGDamia1,

As we are eliminating the dependency on InfluxDB (see #434), @ptomasula and I are considering whether it will be easier to upgrade TSA, which is a completely separate web application and code repository, or recreate it's functionality within the ODM2DataSharingPortal. See:

https://github.com/LimnoTech/WEBTSA/issues/2

So it's helpful to know that having two separate site mapping applications seems a bit confusing and redundant.

SRGDamia1 commented 3 years ago

@aufdenkampe If cost were nothing, I'd prefer to recreate it. There are so many things about webTSA that I'd like to see changed that I hate continuing to patch it.

Prior to maternity leave, I put a lot of effort into data exploration using the HoloViz plotting packages. Relative to the power and interactivity those provide, TSA's capabilities are pretty meager. I have no idea how those work at the production level, but I've been impressed by what I've been able to put together. I know @emiliom has at least heard of them, because I've seen his name in questions on their forum.

emiliom commented 3 years ago

I'm happy to give feedback on holoviz, including Panel "dashboards" (TSA could be viewed as a kind of dashboard). I've used it a fair bit, off and on. I'm currently in the off phase, and need to get back to the on phase very soon.

FYI, here's my holoviz piece de resistance: https://amazon-riverbgc.github.io/TROCAS/docs/databrowser.html It's helpful to read that page before launching the dashboard app. Once you launch it, note that the usage text in the About Data Browser tab is out of date; it doesn't incorporate my last batch of feature enhancements from many months ago.

There are other Python based dashboarding frameworks (Voila, Dash, etc), but I don't have direct experience with them.

aufdenkampe commented 3 years ago

@SRGDamia1, thanks for your thoughts. We're leaning in the direction of scraping TSA, and Scott seemed OK with it. One of the many drivers is that Scott wants to build editing capabilities sometime in 2022. So we know we'll likely be replacing TSA within a year anyways.

The present budget is to just get the full web app working on AWS, and frankly, much of that has been relatively easy (including getting TSA upgraded to Python 3 and Django 2.2 (see https://github.com/LimnoTech/WEBTSA/issues/1). So we were hoping to delay the replacement of TSA until early 2022 with the new batch of funding. However, if just getting TSA to work (without InfluxDB) is a heavy lift, then we might do a minimally functional replacement as a next step.

I've been using HoloViz extensively over the last 9 months for several projects, and we've recently started playing with the HoloViz Panels library, which can be applied to interactive web applications. HoloViz is awesome. @ptomasula also has some JS libraries that he thinks we could leverage. We'll need to explore options.

SRGDamia1 commented 3 years ago

I haven't delved into GeoViews or DataShader, but I've played with all of the other libraries. The interactivity is amazing. I don't have anything publicly hosted to show, but I've been using the param-holoviews-panel combination to make dashboards of all of our DRWI data for others at Stroud to use to look for relationships in the data. Connecting parameters and input data streams is a bit tricky, but once you get it working, the ability to select points on a map and do an on-the-fly regression between parameters at those sites feels pretty cool. I considered trying to run my dashboards on AWS through my personal account, but very quickly decided that it was not going to be possible to get anything going with that quantity of data at free-tier level computing and opted to keep everything on local servers and make users log in and restart the python processes each time.

@dbressler75 would be a key person to get insights from. He's used both MonitorMW's TSA and the python/HoloViz dashboard I've made pretty extensively.

aufdenkampe commented 3 years ago

We have decided to replace TSA with a Time Series Visualization (TSV) page, starting with:

Check out the screenshots I just posted with https://github.com/ODM2/ODM2DataSharingPortal/issues/512#issuecomment-957935317. We think our new TSV addresses this issue.