search

recurve-methods / comparison_groups

Repository for discussion of Comparison Group topics
25 stars 5 forks source link

Have the new methods been tested on periods with no known exogenous factors? #15

Open steevschmidt opened 4 years ago

steevschmidt commented 4 years ago

I haven't reviewed the final document(s) completely (huge!) but I did scan it.

Question: Was there any analysis done on residential exogenous factor predictions during non-COVID periods, as a test of the new methods?

I see some charts with CalTRACK savings estimates on homes between 3/1/20 and 3/19/20, but that period is quite small and it's hard to see if there was any exogenous effect identified.

Is there perhaps a summary chart like Figure 2 (here and on page 9 of the PDF) that covers a longer non-COVID period (e.g. all of 2018 compared to all of 2019) to demonstrate how the new methods work during periods with no known exogenous factor(s)? Seems this would be useful information.

mcgeeyoung commented 4 years ago

With limited time, we decided to focus on the period of COVID impacts, figuring that if we could get results to a reasonable level during this time of high volatility, it would only be better as volatility would decrease. We did some other testing during non-COVID periods to look at underlying uncertainty metrics but the work with a stratified comparison group was done on post-COVID.