Conformal Inference procedure p-value potential issue

[andresnigenda]

Hi, thanks so much for creating this package!

While using this package I uncovered what I think might be a bug in the inference.R conformal_inf() function. I uncovered this potential issue when using a dataset with one single post year.

See this example using the Kansas dataset (restricted to one post intervention year):

• restrict Kansas dataset to only one post observation kansas_1 <- kansas %>% filter(year_qtr <= 2012.25)

• call augsynth asyn <- augsynth(lngdpcapita ~ treated, fips, year_qtr, kansas_1, progfunc = "Ridge", scm = T)

• call summary function with conformal inference procedure to compute p-values summ_kasyn_og <- summary(asyn, inf_type = "conformal")

• p-value by year (one single post year) summ_kasyn_og$att

Time	Estimate	lower_bound	upper_bound	p_val
2012.25	-2.233432e-02	-0.04512445	0.002279013	0.05555556

• p-value for the overall post period (in this case I would expect it to match the by year p-value since we only have one post year) summ_kasyn_og$ average_att

Estimate	Std.Error	lower_bound	upper_bound	p_val
-0.02233432	NA	NA	NA	0.055

After looking at conformal_inf() , compute_permute_ci() gets called using type=="block", whereas compute_permute_pval() inherits type=="iid" from the function's default values. Is this intentional? If I switch the call to compute_permute_pval() to type=="block", then the p-values above match.

[ebenmichael]

No it was not meant to be this way! This is now fixed, with "block" being the default