X must be non-null for matrix completion, but only when bootstrapping

[samzhang111]

Currently, if X is null for the matrix completion estimation, and bootstrapping is performed, then the code crashes with a cryptic Rcpp error:

Error in synth.boot(Y = Y, X = X, D = D, cl = cl, I = I, W = W, EM = EM, :
Bootstrap inference fails. Please check the data.

This happens, for example, on this line if cross-validation is not performed.

On the other hand, when I ran the MC estimation with se=FALSE, this crashing does not occur.

This may be the issue encountered in #65.

[xuyiqing]

Thank you. Does it work with data shipped with gsynth? Could you try inference = "nonparametric"?

On Mon, Sep 27, 2021 at 12:30 AM Sam Zhang @.***> wrote:

Currently, if X is null for the matrix completion estimation, and bootstrapping is performed, then the code crashes with a cryptic Rcpp error:

Error in synth.boot(Y = Y, X = X, D = D, cl = cl, I = I, W = W, EM = EM, : Bootstrap inference fails. Please check the data.

This happens, for example, on this line https://github.com/xuyiqing/gsynth/blob/master/R/core.R#L1581 if cross-validation is not performed.

On the other hand, when I ran the MC estimation with se=FALSE, this crashing does not occur.

This may be the issue encountered in #65 https://github.com/xuyiqing/gsynth/issues/65.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/xuyiqing/gsynth/issues/70, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB2PKGDFFCMEQ63YXHR3SF3UEAMSPANCNFSM5EZ3Y7HQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

-- Yiqing Xu

Assistant Professor Department of Political Science Stanford University https://yiqingxu.org/

[samzhang111]

Yes the issue reproduces with inference="nonparametric", and with the data from the package. Here is an exact line to replicate the issue:

out <- gsynth(Y ~ D, data = simdata, estimator = "mc", index = c("id","time"), se = TRUE, nboots = 50, r = c(0, 5), CV = TRUE, force = "two-way", parallel = TRUE, cores = 4, inference = "nonparametric")

fails, while

out <- gsynth(Y ~ D, data = simdata, estimator = "mc", index = c("id","time"), se = FALSE, r = c(0, 5), CV = TRUE, force = "two-way", parallel = TRUE, cores = 4, inference = "nonparametric")

succeeds. (The difference from the example in the docs is that the formula is Y ~ D instead of Y ~ D + X1 + X2)

[xuyiqing]

Thank you. We'll look into this.

On Mon, Sep 27, 2021 at 1:02 AM Sam Zhang @.***> wrote:

Yes the issue reproduces with inference="nonparametric", and with the data from the package. Here is an exact line to replicate the issue:

out <- gsynth(Y ~ D, data = simdata, estimator = "mc", index = c("id","time"), se = TRUE, nboots = 50, r = c(0, 5), CV = TRUE, force = "two-way", parallel = TRUE, cores = 4, inference = "nonparametric")

fails, while

out <- gsynth(Y ~ D, data = simdata, estimator = "mc", index = c("id","time"), se = FALSE, r = c(0, 5), CV = TRUE, force = "two-way", parallel = TRUE, cores = 4, inference = "nonparametric")

succeeds. (The difference from the example in the docs is that the formula is Y ~ D instead of Y ~ D + X1 + X2)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/xuyiqing/gsynth/issues/70#issuecomment-927625902, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB2PKGE6YOKNHST2KVQ4E4DUEAQIPANCNFSM5EZ3Y7HQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

-- Yiqing Xu

Assistant Professor Department of Political Science Stanford University https://yiqingxu.org/

[shishirdash]

Hello! I ran in to the same issue, curious if there has been any progress on it?

[xuyiqing]

Thank you. We'll look into this.

On Tue, Nov 9, 2021 at 4:42 PM Shishir Dash @.***> wrote:

Hello! I ran in to the same issue, curious if there has been any progress on it?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/xuyiqing/gsynth/issues/70#issuecomment-964674631, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB2PKGBD56LQACD2HR4WXYDULG5WNANCNFSM5EZ3Y7HQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

-- Yiqing Xu

Assistant Professor Department of Political Science Stanford University https://yiqingxu.org/

[PinchOfData]

I ran into the same issue, too.

[PinchOfData]

Any update on this issue?

In the meantime, a simple unorthodox fix is to add a random, uncorrelated variable to the estimation.

data(gsynth)
simdata$X3 <- runif(n=length(simdata$id))
out <- gsynth(Y ~ D + X3, data = simdata, estimator = "mc", index = c("id","time"), se = TRUE, nboots = 50, r = c(0, 5), CV = TRUE, force = "two-way", parallel = TRUE, cores = 4, inference = "nonparametric")

[xuyiqing]

Sorry. Could you send a sample code/data to our email for us to replicate this error?

Best, Yiqing

On Wed, Mar 23, 2022 at 6:01 AM Germain Gauthier @.***> wrote:

Any update on this issue?

— Reply to this email directly, view it on GitHub https://github.com/xuyiqing/gsynth/issues/70#issuecomment-1076348653, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB2PKGD4UNGWWRFNU7AQXWLVBMIZPANCNFSM5EZ3Y7HQ . You are receiving this because you commented.Message ID: @.***>

-- Yiqing Xu

Assistant Professor Department of Political Science Stanford University https://yiqingxu.org/

[PinchOfData]

library(gsynth)
data(gsynth)

# Works with covariates
out1 <- gsynth(Y ~ D + X1 + X2, data = simdata, estimator = "mc", index = c("id","time"), se = TRUE, nboots = 50, r = c(0, 5), CV = TRUE, force = "two-way", parallel = TRUE, cores = 4, inference = "nonparametric")

# Doesn't without
out2 <- gsynth(Y ~ D, data = simdata, estimator = "mc", index = c("id","time"), se = TRUE, nboots = 50, r = c(0, 5), CV = TRUE, force = "two-way", parallel = TRUE, cores = 4, inference = "nonparametric")

Will send it by email, too.

(great package btw)

[xuyiqing]

Thanks! Licheng, could you take a look?

On Wed, Mar 23, 2022 at 10:25 AM Germain Gauthier @.***> wrote:

library(gsynth) data(gsynth)

Works with covariates

out1 <- gsynth(Y ~ D + X1 + X2, data = simdata, estimator = "mc", index = c("id","time"), se = TRUE, nboots = 50, r = c(0, 5), CV = TRUE, force = "two-way", parallel = TRUE, cores = 4, inference = "nonparametric")

Doesn't without

out2 <- gsynth(Y ~ D, data = simdata, estimator = "mc", index = c("id","time"), se = TRUE, nboots = 50, r = c(0, 5), CV = TRUE, force = "two-way", parallel = TRUE, cores = 4, inference = "nonparametric")

Will send it by email, too.

— Reply to this email directly, view it on GitHub https://github.com/xuyiqing/gsynth/issues/70#issuecomment-1076604768, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB2PKGDLZSSMWDUTW4USLTDVBNHYRANCNFSM5EZ3Y7HQ . You are receiving this because you commented.Message ID: @.***>

-- Yiqing Xu

Assistant Professor Department of Political Science Stanford University https://yiqingxu.org/

[mjpavlik]

Bumping this issue, as I am getting the same error. Thanks so much for this amazing package and for mc integration :)

[xuyiqing]

Sorry for the deley. We'll get to it soon!

On Wed, May 11, 2022 at 11:37 AM mjpavlik @.***> wrote:

Bumping this issue, as I am getting the same error. Thanks so much for this amazing package and for mc integration :)

— Reply to this email directly, view it on GitHub https://github.com/xuyiqing/gsynth/issues/70#issuecomment-1124163398, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB2PKGFFGMKRCF37TKPWRC3VJP46LANCNFSM5EZ3Y7HQ . You are receiving this because you commented.Message ID: @.***>

-- Yiqing Xu

Assistant Professor Department of Political Science Stanford University https://yiqingxu.org/

[ChrisDoll]

Adding another workaround, reverting to version 1.1.4

# devtools
install.packages("devtools", version = "2.2.2", repos = 'http://cran.us.r-project.org', type = "source")

# Install for functioning commit: Aug 26, 2019, 1.1.4
devtools::install_github("xuyiqing/gsynth", ref = "461a761")

The error seems to be caused by:

               X.boot <- NULL
                if (p > 0) {
                    X.boot <- X[,boot.id,,drop = FALSE]
                }

in 'core.R'. When and why this is the case I don't know.