carloscinelli / sensemakr

Suite of sensitivity analysis tools for OLS
https://carloscinelli.com/sensemakr/
83 stars 15 forks source link

Spot check on interpret printout #27

Closed statsccpr closed 5 years ago

statsccpr commented 6 years ago

In the vignette https://github.com/chadhazlett/sensemakr/blob/master/vignettes/sensemakr.md

edit the worstcaseinterpret() matches graphic of worstcaseplot() one parameter is x axis and other parameter is curve type

@chadhazlett

The two different non-symmetric sentence structures is intended right? Along with 'two dimensions', intended to provide users with two grammatical variants?

Result of interpret

## 
## ### Benchmarking ###
## 
## ---Using the covariate most strongly associated with the treatment assignment as a benchmark---
## 
## An unobserved confounder explaining as much of the treatment as 'female' (0.009)  would be able 
to cause at most a bias of 0.062 with an adjusted treatment effect of 0.035 in the extreme case where 
the confouder explains all the residual variance of the outcome (R2y = 1).
## 
## ---Using the covariate most strongly associated with the outcome as a benchmark---
## 
## An unobserved confounder as associated with the outcome as 'female' (R2y = 0.109)  would have to 
be at least 18.7 times as strongly associated with the treatment (reaching R2d = 0.17) in order to reduce 
the treatment effect by 100%
chadhazlett commented 6 years ago

Some of my changes to the language seem to have been lost. I can revise and then let's see what everybody thinks.

On Sep 25, 2017 3:25 PM, "statsccpr" notifications@github.com wrote:

In the vignette https://github.com/chadhazlett/sensemakr/blob/ master/vignettes/sensemakr.md

Just checking, is this order right? @carloscinelli https://github.com/carloscinelli The printout (perhaps correct and my reaction is wrong) has 100 paired with 2 and 25 paired with 8

My initial reaction was they should be monotonically increasing, say 100 with 20 and 25 with 4

But, the code looks right.

Considering the extreme scenarios of unobserved confounders explaining 100%, 25% of the residual

variance of the outcome, they would have to, respectively, explain at least 2.19%, 8.21% of the treatment assignment to reduce the treatment effect in 100%.

This is from worstcaseinterpret The relevant portion is below generates the output above

r2dc = t^2/(t^2 + (scenarios/q^2)*df)

cat("Considering the extreme scenarios of unobserved confounders explaining ", paste0(scenarios100, "%", collapse = ", "), " of the residual variance of the outcome, they would have", " to, respectively, explain at least ", paste0(round(r2dc100, 2), "%", collapse = ", "), " of the treatment assignment to reduce the treatment effect in ", round(q*100, 2), "%.", sep = "")

@chadhazlett https://github.com/chadhazlett

The two different non-symmetric sentence structures is intended right? Along with 'two dimensions', intended to provide users with two grammatical variants?

Result of interpret

Benchmarking

---Using the covariate most strongly associated with the treatment assignment as a benchmark---

An unobserved confounder explaining as much of the treatment as 'female' (0.009) would be able

to cause at most a bias of 0.062 with an adjusted treatment effect of 0.035 in the extreme case where the confouder explains all the residual variance of the outcome (R2y = 1).

---Using the covariate most strongly associated with the outcome as a benchmark---

An unobserved confounder as associated with the outcome as 'female' (R2y = 0.109) would have to

be at least 18.7 times as strongly associated with the treatment (reaching R2d = 0.17) in order to reduce the treatment effect by 100%

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/chadhazlett/sensemakr/issues/27, or mute the thread https://github.com/notifications/unsubscribe-auth/AFVCyDYoEXO_0rGpMOptspf1XPwMvZVaks5smChVgaJpZM4PjbNQ .