nycehs / data-features

Hub pages and data features for the EH Data Portal
0 stars 0 forks source link

error handling on location selection 18-value test #17

Closed mmontesanonyc closed 2 years ago

mmontesanonyc commented 2 years ago

I think our error handling isn't quite working as expected:

image

Looks like valid_vals doesn't quite work - it still finds 24 valid values, even in this instance where the Broadway monitor is down.

image

It then tries to average the most recent 24 hours of data, but can't, so avg_24 = NaN instead of null.

So I think a couple things need to happen - troubleshooting the valid_vals issue, and then improving the error handling so that if we don't have 18+ values, then we print a message in place of the data output - something like, "No value: sometimes monitors go down or have other problems. We only produce average values if there are more than 18 hourly readings over the last 24 hours."

cgettings commented 2 years ago

Looks like NA values are coming in as 'NA' strings

mmontesanonyc commented 2 years ago

so the valid_vals bit might need a test to count the numerical values instead of counting the values.

cgettings commented 2 years ago

I'm trying to count the output from a logical test, but I can't quite get it to work. In R, I would use count(arqTable, locSelect == 'NA'), and then use the number of TRUEs, but no translation of that has worked so far.

cgettings commented 2 years ago

@mmontesanonyc arq_NA_wide_test.html in the hotfix-NA-handling branch shows a working fix for this issue. I'm going to edit realtime.html, then I'll create a PR.

mmontesanonyc commented 2 years ago

Looks great! Seems like you're getting a nice handle on Arquero. And, the code is really well documented - I like the step by step console logs, which helped me follow along with what you were doing.