Calculate historical ranges and add them to time series plots

[daviswx]

Once the ranges of historical variability are calculated, add them (as whiskers?) to the time series plots and add horizontal lines across the time series corresponding to these ranges.

Here's an example of adding horizontal lines in Highcharts: http://jsfiddle.net/wergeld/hXDYh/2/

[hadinon]

Once this is implemented, add a historical avg tooltip to timeseries legend.

[hadinon]

Quick update on this -- all historical avg +/- 2 standard deviations maps have been generated for:

• All thresholds (32F, 28F, 25F, 20F, 15F) of Extreme Min Temp tool
• Summer Avg Temperature

Summer Precip maps are being generated for this and should be done today.

[daviswx]

Suggestions from today's meeting about implementing the historical standard deviation maps:

• Use a different colored map title bar (maybe blue, to match the time series) for the historical data
• Change the map display option, middle map title, and time series legend title to "Historical Observed" (instead of "Historical Average")
• Possible side map title wording: "Lowest/Highest Likely Observed" or "Lowest/Highest Typical Observed" or "Lowest/Highest Expected Observed" or "Lowest/Highest Commonly Observed"
• Use error bars in the time series plot

[daviswx]

The new map title style and wording for the historical display is now in place.

The Extreme Min Temp tool has the standard deviation side maps working. I will get those added in for the other tools.

After that, I will add the historical error bars to the time series.

[daviswx]

The Summer Temperature and Summer Precipitation tools now have the historical standard deviation maps added in as well.

[hadinon]

Looks good! A few comments:

1) Should we adjust the titles on the maps and time series? A new map title is easier for me to brainstorm and could be: "Historical Observed Number of Days Per Year with Minimum Temperatures <32F". For me, the time series title is tougher to rename because the projected values shown are averages but the historical values shown are not (or will not be, since they are coming soon) all averages. Thoughts?

2) I'm sure you are waiting to update the time series legend title but wanted to give a quick reminder about changing that to, "Historical Observed" (instead of "Historical Average").

3) Have you updated the functionality for the location selection to return the correct values for each map? Currently, it appears that the same value is being populated on all 3 maps. I remember this was one of the last items implemented during the development of one of the tools so if you're just not there yet that's totally fine.

[daviswx]

Good point about the title wording. For all of the climate tools' map titles, I'm fine with saying "Historical Observed [XYZ]". For the time series titles, part of me thinks we could get away with keeping the titles as-is ("Average [XYZ]") since the historical average is shown on there, and the projections are for the change in the average. The fact that the range of historical variability is also shown can be noted in the legend.

Also, thinking ahead to when we add the model error, that could complicate the title wording even more if we wanted to be really pedantic about saying exactly what is being shown, so IMO keeping it simpler is better. Maybe we can discuss this at the PINEMAP meeting today.

Speaking of that legend wording, yes, it is on my to-do list. What should this wording say? "Historical Observed"? "Historical Variability"? And should there be a tooltip next to it with additional explanation?

Also, the historical error bar in the time series isn't yet dynamic, but I'm working on that now. I'll try to have it working before today's meeting.

[hadinon]

For the record, wanted to write an update based on our conversation at last week's PINEMAP meeting: 1) yes, keep wording "as is" on the time series titles and note the range (or would it be spread?) of historical variability in the legend tooltip 2) change legend wording to say, "Historical Observed" and add a tooltip

[daviswx]

I think everything is now updated to show the historical ranges. Here is what I have changed for the climate risk tools:

• The map titles now say "Historical Observed..." instead of "Historical Average...".
• The three-map layout is used for the Historical Observed display.
• The time series shows the historical ranges on error bars for the Historical Observed and Projected Average displays. These ranges now update dynamically.
• For those displays, the time series legend includes an updated image, label, and tooltip. For the Projected Change display, the legend is the same.
• The time series tooltips now show the historical range when hovering over that column/error bar.

I think that takes care of everything related to the historical ranges, but we may want to review all 3 tools to make sure they seem to be working correctly. If all looks good, we can close out this issue and I will commit these changes and pull them down on the public DSS site.

[hadinon]

Updates all sound good.

OK, so I know why we're doing this but I just wanted to point out that it may be potentially confusing to a user to have the time series labeled as Historical Average (when you hover over) and the center map labeled as Historical Observed (see below screenshot). I know that users should understand that these are the same but I'm not 100% sure they will make the connection. So, would you be opposed to changing the center map title back to Historical Average to match the verbiage in the time series?

[daviswx]

I was thinking the same thing as I was updating the time series. Yes, I agree that the center map should be titled "Historical Average". I just made that change on the climate risk tools.

[hadinon]

Perfect, thanks. Let me take a more detailed look at all 3 tools to make sure they're working properly. Then, it is probably ready to push to public!

[hadinon]

Hmmm looks like the "whiskers" are still showing up on this time series plot. Thoughts?

[daviswx]

I just noticed that as well and made a quick change that should fix it (for the Projected Change plots, I'm just setting the upper/lower bounds both to zero). Let me know if you're still seeing this.

[hadinon]

Yep, looks like it went away! I'll keep checking :smile:

[hadinon]

OK, this is a ridiculous, extreme case (who the heck in FL would look at a threshold of 15F) but the whiskers don't appear in this example because the lowest likely observed is 0.0 which makes sense but are you OK with it not showing up in that case? Also, I just noticed that the lowest likely observed map is not labeled with a 0.0 days. Thoughts?

[daviswx]

For values near zero, or very close to the historical average, I think it's fine if the error bars aren't actually visible. The only way around this, I think, would be to remove the thicker blue bar entirely, and even then, the error bars would probably be a single blip at 0, so I'm not sure if that would add any extra information.

This does make me think of another question, though. Should we add labels on the whiskers to actually show the values on the upper and lower ends of the range? Currently, we're just labeling the historical average, but I think it might be useful to see the extremes, too.

This could again be a problem when the range of values is really small. As an example, pretty much everywhere on the Summer Temperature has a pretty narrow range, so the top label might overlap with the historical average one.

I'm fine if we want to hold off on making a decision about that and just leave things as-is for now.

[hadinon]

Hmmm yes, agreed. Let's plan to bring these two items up with Ryan at our PINEMAP meeting today.

Are you able to add a label to the left map (Lowest Likely Observed) for the above example to say 0.0?

[hadinon]

Ohhh I think the problem is the value is negative. Hmmmmm I'm not sure how you would handle that because aren't there also -999's sometimes and we wouldn't want those to be labeled, right?

[daviswx]

I'll look into it and try to get it fixed. In my OPeNDAP calls, I am setting negative values to zero, so it's probably just ignoring the zero value because it thinks it's null or something.

[daviswx]

That example with northern Florida in the Extreme Min Temp tool is working for me now. JavaScript was treating 0 values as undefined, but I found a workaround that checks the variable type (if it's 0, it still recognizes it as a number rather than a null/undefined variable).

I did notice that clicking in south Florida gives a message that complete time series data is unavailable. I think that's because all of the historical and future values are zero. I will check the code and see if I can add some similar statements so at least zero values show up on the map and in the time series.

[hadinon]

Yes, it is fixed. However, I think I remember why you did that... so that the values over the ocean would not show up because now when I click over the ocean they show up. This is frustrating! :angry:

[daviswx]

Yes indeed. I'm running into that same issue trying to fix it. I'm sure I can work it out with some more time, but if we want to roll out the latest changes today, it may be best to leave it as it was for now (so spots over land with all zero values show up as invalid) and fix it later. I can create a separate issue about that with some more details.

[hadinon]

Yep, that sounds like a good plan. Thanks!

[daviswx]

Just to summarize our remaining action items related to the historical ranges:

• Discuss whether the historical observed error bars should have labels
• Fix the issue with data points over land showing as invalid (issue #79)

[daviswx]

Updates from today's meeting:

• Change "Likely" to "Typically"
• Make the historical average box a thinner line
• Do NOT label the error bars; those are not part of the average that the projections are modeling

[daviswx]

These updates are done. I think those were our only remaining issues with the historical ranges, so I will close this issue. If I forgot something or if anything comes up later, feel free to re-open it.

[daviswx]

At today's meeting, we decided to turn off the historical ranges on the maps and time series and show only the historical average. This should remove any confusion with having the projections be just for the long-term average while the historical data captures more of the year-to-year variability.

[daviswx]

The historical side maps have been turned off. Should we change the menu and title to say "Historical Average" instead of "Historical Observed"? Historical observed is technically correct, but using the term "average" may better agree with the title "Projected Average".

I still need to remove the historical range from the time series, legend, and any references to it in the tooltips/FAQs.

[daviswx]

All references to the historical range/spread/variability and side maps should be gone at this point. Let me know if you notice any others.

[daviswx]

Since we haven't found any other references to the historical side maps or bugs with them not being there anymore, I am closing this issue.