JeffersonLab / wave

Web Archive Viewer and Expositor
https://epicsweb.jlab.org/wave
MIT License
2 stars 0 forks source link

Data change when the start of the window changes #18

Closed nsjarvis closed 1 year ago

nsjarvis commented 1 year ago

There is something wrong with the data at the start of this plot. (1) These measurements never have standard deviation = 0. (2) Next, if I change the start time to 01:00 or later then the variations at 2am disappear.
I'm not sure if I'm making some silly mistake or there's a problem in EPICS.

https://epicsweb.jlab.org/wave/?start=2023-01-17+00%3A00%3A00&end=2023-01-17+11%3A00%3A00&myaDeployment=&myaLimit=100000&windowMinutes=30&title=&fullscreen=false&layoutMode=3&viewerMode=1&pv=CDC%3Ahv%3AA%3A1%3Aimon&pv=IBCAD00CRCUR6&IBCAD00CRCUR6yAxisLabel=&IBCAD00CRCUR6yAxisMin=0&IBCAD00CRCUR6yAxisMax=1000&IBCAD00CRCUR6yAxisLog&IBCAD00CRCUR6scaler=&CDC%3Ahv%3AA%3A1%3Aimonlabel=CDC%3Ahv%3AA%3A1%3Aimon&CDC%3Ahv%3AA%3A1%3Aimoncolor=%231f78b4&CDC%3Ahv%3AA%3A1%3AimonyAxisLabel=&CDC%3Ahv%3AA%3A1%3AimonyAxisMin=&CDC%3Ahv%3AA%3A1%3AimonyAxisMax=&CDC%3Ahv%3AA%3A1%3AimonyAxisLog=&CDC%3Ahv%3AA%3A1%3Aimonscaler=&IBCAD00CRCUR6label=IBCAD00CRCUR6&IBCAD00CRCUR6color=%23b2df8a

slominskir commented 1 year ago

A good way to sanity check is to compare with the desktop version, which looks like this:

image

The link you provided results in:

Chart (5)

That is clearly not matching. However, I noticed you've manually set the y-axis for IBCAD00CRCUR6 to extend to 1000. You left CDC:hv:A:1:imon to autoscale (no max y-axis). If you remove the max y-axis and allow both to autoscale you'll get a reasonable match to the desktop viewer:

image

nsjarvis commented 1 year ago

This is what I see using the link in my first post Chart (4)

nsjarvis commented 1 year ago

and when I change the x-axis to start at 2am, I get the following Chart (5)

nsjarvis commented 1 year ago

Could this be a browser compatibility issue? I'm using a chromebook (because I'm far from home and it does not weigh much).

slominskir commented 1 year ago

Could be a timezone issue if you're in a different timezone.

slominskir commented 1 year ago

Try:

  1. Using a different web browser.
  2. Tunneling/VPN into the JLab network and using a web browser inside (will be US New York Eastern Standard Time)
  3. Check browser console for errors (F12 key). Are there any non-update events such as NaN events?
  4. Check to make sure nothing unusual about parameters in the URL. Perhaps start with a fresh WAVE window and build chart from scratch.
slominskir commented 1 year ago

More things to check:

WAVE queries the archiver via myquery here: https://epicsweb.jlab.org/myquery/interval-form.html

The query for CDC:hv:A:1:imon should be (F12 developer console network tab should show): https://epicsweb.jlab.org/myquery/interval?c=CDC:hv:A:1:imon&b=2023-01-17+00:00:00&e=2023-01-17+11:00:00&u=&p=&v=17&m=&l=100000

The result is JSON format and documented here: https://github.com/JeffersonLab/myquery/wiki/API-Reference#multiple-event-query-interval. I wonder if appending &a= to the end of the myquery URL fixes the issue?

nsjarvis commented 1 year ago

Thanks for the suggestions.

4 Create the url afresh - I did this already, before creating the issue. Originally I was plotting 5 things, wondered if it was too many.

5 Append &a= This doesn't make any difference.

I'll try the other ideas out when I get back to the US next week. I don't have a F12 key. I'm using an
Asus C223NA-DH02 Chromebook It has been fine for everything else (except with root I have to save the canvas to a file & then scp it, this is a pain but still beats carrying a 3lb macbook around).

nsjarvis commented 1 year ago

This is the start of the page returned by your query 2 comments above.

{"datatype":"DBR_DOUBLE","datasize":1,"datahost":"opsmya0","ioc":"cagwspare","active":true,"sampled":false,"count":30926,"data":[{"d":1673931600000,"v":10.100000381469727},{"d":1673931600212,"v":10},{"d":1673931600877,"v":9.899999618530273},{"d":1673931601548,"v":0.6000000238418579},{"d":1673931602254,"v":0.5},{"d":1673931602926,"v":0.10000000149011612},{"d":1673931603601,"v":0.20000000298023224},{"d":1673931604931,"v":0.4000000059604645},{"d":1673931605707,"v":0.10000000149011612},

nsjarvis commented 1 year ago

I think that your timezone idea was correct. I'm in Chile, 2h ahead. The plot is in local time, the steps in the CDC variable just before "10am" were during its HV scan just before 8am JLab time. I think the gui is trying to cope with the extra 2h by making the first 2h of the plot constant. This explains why I always see the first 2h of the plot constant, whatever time window I ask for.
Would it be possible to force the wave gui to always display the x axis in JLab time, please? That would be more useful, especially when trying to correlate the plot with the logbook.

slominskir commented 1 year ago

Yeah, that myquery data is what I get too. Timezone issue sounds likely. Using https://www.epochconverter.com/ with first "d" date of 1673931600000 is a UNIX timestamp that maps to Jan 17 00:00:00 GMT-05:00 (New York / Eastern Standard Time). That's correct when viewed in a browser in EST, but depending on the timezone your web browser detects that probably maps to something else, which may explain the difference. The start and end timestamps in the myquery data request always assume EST timezone. That's probably what we always want. It appears the x-axis bounds are possibly always fixed in EST too whereas the UTC data points are drawn in user local time (GMT-3, for Chile in your case). Either the bounds need to adjust to user's browser local timezone too to match the data or else I think always displaying the data points themselves in EST/EDT make sense.

The later is probably best, but Timezone handling is insidious in web browsers because they don't have an IANA timezone database built-in - you can't just hand it UTC dates and ask the web browser to convert dates to someone else's timezone or even to past / future times in the browser local zone (without a third party JavaScript timezone database library, that you then must keep up-to-date and that negatively affects page load). One solution is to instruct browsers (and graphing libraries) to do no conversions at all in browser and just display UTC dates, but actually provide the "UTC dates" shifted -5:00 for EST (make server do timezone shifting). I thought that was how WAVE was setup. Apparently not as the "a" option isn't present and timestamps are actual UTC, not server shifted UTC. Maybe I broke this recently doing housekeeping changes, but in any event I'll do some more digging. Thanks for chasing this down on your end.

slominskir commented 1 year ago

Duplicate: https://github.com/JeffersonLab/wave/issues/11