search

LowellObservatory / Camelot

'Tis a silly place. Repository for one offs and toy models exploring a random assortment of topics
Mozilla Public License 2.0
0 stars 0 forks source link

Investigate using 'RemoteSource' for Bokeh plotting #6

Closed astrobokonon closed 5 years ago

astrobokonon commented 5 years ago

I noticed that the Bokeh documentation lists 'RemoteSource' as a data source; see here for the relevant documentation.

I sorta wonder if I could just figure out how to give it the correct influxdb query string and this would take care of the rest. It could be a huge complexity saving on the backend! But, I'm sorta already down the path of doing it myself manually, so this is probably a better "beta" target.

astrobokonon commented 5 years ago

Note for the future: it's really seeming like AJAXDataSource or example here is the way to try; seems to require a CDS-style JSON response, and I'm not sure how close influx's reply will be to that.

I also don't know how to maintain the 'statefulness' or consistency of the data this way; seems like I'd still want to only query everything once and then update much like I'm doing manually right now.

astrobokonon commented 5 years ago

Thinking about bumping this up pretty significantly in the overall roadmap - I'm running into problems streaming multiple updates to the ColumnDataSource for weather, for example.

When "series A" updates at a different interval than "series B" and I want to plot updates for both of them, using a ColumnDataSource I have to somehow make them the same length at all times - that's kinda a pain, though not insurmountable.

If "series A" has an update with 5 points in 7 columns, and "series B" has 2 updates in 1 column, then I could just do what Grafana/Influx seem to do and fill in "series B" will null values that aren't plotted or something similar. But, I've gotta roll that logic myself or alter my query structure to make Influx do it for me.

The latter might be easier (better Influx query) than the former, but I don't know yet. Can one make a single influx query to two different databases and still use the magic fill() at the end to make influx put everything on a common/consistent timebase? If that's possible, that would potentially save lots of time on getting this all up and running.

astrobokonon commented 5 years ago

To the final point above; this is a valid query

SELECT 100-"cpuIdle", "barPressure_mbar" AS "mean_cpuIdleDiff", "pressure" FROM "DCTInstrumentHosts"."autogen"."MachineStats", "DCTInfo"."Hold6w"."WRS.WRSPubDataSV.WRSDataPacket" WHERE time > :dashboardTime: FILL(null)

(at least in telegraf, using the template variable for time)

Gets a little more complicated when trying to filter via tags, but probably do-able as well.

astrobokonon commented 5 years ago

I'm actually deciding to swing back in the other direction, and keep what I've got for now. I've found acceptable (to me, at least) abstractions to simplify and unify the plots as much as I can, and while it's a little complicated I think it's reasonable. Will reopen this if needed in the future but I think we're good thru at least the beta version.