klarsen1 / MarketMatching

Other
131 stars 37 forks source link

How to get value after which the response value would have been siginificant? #29

Closed frieroijers closed 10 months ago

frieroijers commented 1 year ago
library(MarketMatching)
data(weather, package="MarketMatching")
mm <- best_matches(data=weather,
                   id_variable="Area",
                   date_variable="Date",
                   matching_variable="Mean_TemperatureF",
                   parallel=FALSE,
                   warping_limit=1, # warping limit=1
                   dtw_emphasis=1, # rely only on dtw for pre-screening
                   matches=5, # request 5 matches
                   start_match_period="2014-01-01",
                   end_match_period="2014-10-01")

results <- MarketMatching::inference(matched_markets = mm,
                                    test_market = "CPH",
                                    end_post_period = "2015-10-01")

pred = sum(results$Predictions$Predicted)
actual = sum(results$Predictions$Response)
lower = sum(results$Predictions$lower_bound)
upper = sum(results$Predictions$upper_bound)
effect = actual-pred

Is it possible to determine at what cumulative response value the response would have been significant?

Let's say I have a predicted cumulative value of 19007. The confidence interval is [16053, 21967]. The cumulative response value is 18497, and it not significant with 95% confidence interval. This makes sense ofcourse since the response is lower than the prediction. However, I want to know how far off the response value was from reaching statistical significance. Is this value presented in the package output? Is it maybe as easy as taking the sum of the upper bound values?

klarsen1 commented 10 months ago

Not exactly what you're asking -- I typically look at the probability of causal impact to gauge. The tail probability is part of the output.