I have a problem interpreting the outputs from microsynth.
The code below is used to test the impacts of forest conservation incentives on annual forest cover ("fc"), adjusting by propensity score-style weights for pre-program outcomes and annual rainfall, and other time-invariant confounding variables affecting forest cover.
library(microsynth)
set.seed(1234)
foo <- readRDS(url("https://github.com/sebbych/scm/raw/main/foo.RDS"))
#without jackknife and permutation
ms_foo <- microsynth(foo,
idvar="Id", timevar="year", intvar="treat",
start.pre=2001, end.pre=2007, end.post=2019,
match.covar.min=c("mean_elev","mean_slope", "access00","mean_pop05","mean_crop", "dist_roads"),
match.out.min="rainfall",
result.var="fc",
test="lower",
check.feas=TRUE, use.backup = TRUE,
#jack=T,perm=1000,
n.cores = 1 #!!!! crashes if i use the the 10 cores of my virtual machine...
)
summary(ms_foo)
plot_microsynth(ms_foo,plot.var = 'fc',all=NULL,
start.pre=2000, end.pre=2007, end.post=2019,
main.tc="levels",main.diff="difference Treatment-Control",
ylab.tc="tree cover")
I am confused because the outputs of microsynth is
Results:
end.post = 2019
Trt Con Pct.Chng Linear.pVal Linear.Lower Linear.Upper
fc 16685537400 15712329552.63 6.2% 0.9996 3.0% 9.5%
Pvalue indicates statistical non-significance, but the pct change seems to be within confidence intervals. So, should i assume statistical non-significance despite the confidence interval not containing zero?
When I include jackknife and permutations (not included in the code because files is too heavy for github), i also get a similar divergence for the permutation (Jackknife seems ok)
end.post = 2019
Trt Con Pct.Chng Linear.pVal Linear.Lower Linear.Upper Jack.pVal Jack.Lower Jack.Upper
fc 16685537400 15751022222.96 5.9% 0.9998 3.0% 8.9% 0.6826 -13.8% 30.1%
Perm.pVal Perm.Lower Perm.Upper
fc 0.9990 2.7% 9.6%
I am surprised about the permutation, as it seems the difference identified in the plot is quite unique.
Can you confirm that I am interpreting outputs correctly and that I should stand by the diagnostic on the Pvalue rather than confidence intervals, even when they diverge? or trusting more the jackknife over other divergent statistics?
Thank you for any tips in interpreting these outputs!
PS:I also get an error on my Windows 10-core virtual machine when I try to parallelize the calculation. Setting n.cores at a different value than 1 yields the following error message:
"Error in parallel::parLapply(cl = cl, X = (for.max + 1):tot.col, get.stats1.sub, :
object 'reps' not found"
I assume it is a problem of the Parallel library rather than Microsynth. I am happy with the way the package work with 1 core, so everything all right. But if others have found a solution to this issue, I am interested!
Hi,
I have a problem interpreting the outputs from microsynth. The code below is used to test the impacts of forest conservation incentives on annual forest cover ("fc"), adjusting by propensity score-style weights for pre-program outcomes and annual rainfall, and other time-invariant confounding variables affecting forest cover.
I am confused because the outputs of microsynth is
Pvalue indicates statistical non-significance, but the pct change seems to be within confidence intervals. So, should i assume statistical non-significance despite the confidence interval not containing zero?
When I include jackknife and permutations (not included in the code because files is too heavy for github), i also get a similar divergence for the permutation (Jackknife seems ok)
I am surprised about the permutation, as it seems the difference identified in the plot is quite unique.
Can you confirm that I am interpreting outputs correctly and that I should stand by the diagnostic on the Pvalue rather than confidence intervals, even when they diverge? or trusting more the jackknife over other divergent statistics?
Thank you for any tips in interpreting these outputs!
PS:I also get an error on my Windows 10-core virtual machine when I try to parallelize the calculation. Setting n.cores at a different value than 1 yields the following error message: "Error in parallel::parLapply(cl = cl, X = (for.max + 1):tot.col, get.stats1.sub, : object 'reps' not found"
I assume it is a problem of the Parallel library rather than Microsynth. I am happy with the way the package work with 1 core, so everything all right. But if others have found a solution to this issue, I am interested!