Speed up hydrograph render

UGS-GIO / groundwaterMonPortal

UGS Groundwater Data Portal web application that compiles a large data set of groundwater data and displays it on an interactive web map

https://apps.geology.utah.gov/gwdp

6 stars 0 forks source link

Speed up hydrograph render #17

Closed inkenbrandt closed 2 years ago

inkenbrandt commented 3 years ago

Why do you feel this application needs what you're requesting? Most charts take more than three minutes to render, making the end user assume that the functionality is absent. For example, the hydrograph of site PW02A

Describe the solution you'd like and why it builds value There are a few potential solutions. Here are a couple of my ideas to speed up charts:

See if there is a process that is causing the lag
Change data resolution based on zoom level: https://www.amcharts.com/docs/v3/tutorials/dynamically-changing-granularity-data-based-zoom-scope/
figure out how the USGS is doing it so well: https://waterdata.usgs.gov/monitoring-location/09380000/#parameterCode=00060&period=P1Y&compare=true

Additional context Add any other context or screenshots about the feature request here.

marshallrobinson commented 3 years ago

I've noticed this happening for some time, and have wondered what is the cause of it. They weren't always going so slow. Is there additional data that the chart is consuming and trying to display now?

Thanks for the possible solutions.

inkenbrandt commented 3 years ago

No additional data that I am aware of. The only new data is when we append new water level records as we collect them, but the example that I tried hasn't had new records since 2017.

marshallrobinson commented 3 years ago

Good to know. It could be something to do with the server the data is hosted on, or the server the application's code is hosted on. My initial hunch is that it is more likely something to with the server the code is hosted on because it shares resources with many other applications.

inkenbrandt commented 3 years ago

Is there a way we can test your hunch? I think you might be right because it is a little laggy when rendering the sites on the map as well.

marshallrobinson commented 3 years ago

Possibly, let me think about it and get back to you.

-Chrome's network tab would be a good place to start to see which request is going the slowest and where it is getting sent.

leeoniya commented 3 years ago

yes, it's waiting for 40s to receive 234kb of data (~30k datapoints). amcharts is quite slow, but that's not the main issue here.

bench & shameless plug: https://github.com/leeoniya/uPlot#performance

slow-net

inkenbrandt commented 3 years ago

Most hydrographs won't even plot now.

marshallrobinson commented 3 years ago

@lance-weaver do you have a chance to troubleshoot this issue in the next few weeks?

inkenbrandt commented 3 years ago

ArcGIS Server limits the display to 22,000ish records. @marthajensen created a dynamic view that aggregated water level records to daily data. However, ArcGIS Server crashes when trying to render this view in the charts. The only GIS data on this portal is the point data on the map. There is no need to use an ESRI product to chart non-geographic database data.

marthajensen commented 3 years ago

The dynamic view was created in PostgreSQL - which may be why ArcGIS Server doesn't load it. The remaining options are to create the view in ArcGIS (query layer) or make the aggregated table a static table. If I create a static table from the aggregated table, I would update it by running a python script each time @inkenbrandt adds records to the water level table.