search

xuyiqing / gsynth

Generalized Synthetic Control Method
Other
132 stars 40 forks source link

Error in cumuEff(): object 'D.boot' not found #30

Open michaeldgarber opened 4 years ago

michaeldgarber commented 4 years ago

Dear @xuyiqing, @liulch, I would like to implement your function for finding cumulative effects (cumEff) but I get the following error when running this line of code. The gsynth object was successfully created.

gsynth_cum<- gsynth::cumuEff(gsynth_object,
                                cumu= TRUE,
                                id = NULL,
                                period = NULL)

Error in gsynth::cumuEff(gsynth_object, cumu = TRUE, id = NULL, period = NULL) : object 'D.boot' not found

Thanks for any help you may be able to offer and for the terrific package. -Michael

liulch commented 4 years ago

Hi @michaeldgarber , can you also add the code that creates the gsynth object? Thx!

michaeldgarber commented 4 years ago

Hi @liulch - thanks for your reply. Here's a reprex, using the turnout data, which returns the same error for me. Perhaps I need to specify something in the 'id' and 'period' arguments?

gsynth version ‘1.1.4’ R version 3.6.1 (2019-07-05)

library(gsynth)
data(gsynth)

Use the turnout data, which has differential timing of treatment. Inspect data

panelView(turnout ~ policy_edr, data = turnout,  index = c("abb","year"), 
pre.post = TRUE, by.timing = TRUE) 

out <- gsynth::gsynth(turnout ~ policy_edr + policy_mail_in + policy_motor,
              data = turnout,  index = c("abb","year"), se = TRUE, 
               inference = "parametric", r = c(0, 5), 
              CV = TRUE, force = "two-way", nboots = 1000, seed = 02139)

out_cum_eff <- gsynth::cumuEff(out,
        cumu = TRUE, 
         id=NULL,
       period = NULL)

Error in gsynth::cumuEff(out, cumu = TRUE, id = NULL, period = NULL) : object 'D.boot' not found

michaeldgarber commented 4 years ago

Hi @xuyiqing and @liulch ,

I see that you have added eff.boot as an output of the gsynth function. Thank you. As a result, I no longer need thecumuEff function, as I can calculate everything I need using the output of gsynth. So, for my purposes, the issue here could be closed, but I'll leave it open since, technically, I think the issue is still present.

Thanks again, Michael

xuyiqing commented 4 years ago

Dear Michael,

We'll try to address your suggestion during the holidays (perhaps in Dec). I'm finishing up this quarter (teaching two courses). I hope it'll be fairly soon. Thanks for your suggestions again!

Best, Yiqing

On Mon, Nov 18, 2019 at 3:43 PM Michael D Garber notifications@github.com wrote:

Hi @xuyiqing https://github.com/xuyiqing and @liulch https://github.com/liulch ,

I see that you have added eff.boot as an output of the gsynth function. Thank you. As a result, I no longer need the cumuEff function, as I can calculate everything I need using the output of gsynth. So, for my purposes, the issue here could be closed, but I'll leave it open since, technically, I think the issue is still present.

Thanks again, Michael

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/xuyiqing/gsynth/issues/30?email_source=notifications&email_token=AB2PKGFQ6BYQP46QCBJR6ILQUMSB3A5CNFSM4I5GOY32YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEMKHRI#issuecomment-555262917, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB2PKGATIQ5GDCQ6ZFET2BTQUMSB3ANCNFSM4I5GOY3Q .

-- Yiqing Xu

Assistant Professor Department of Political Science University of California, San Diego http://yiqingxu.org/

michaeldgarber commented 4 years ago

No problem! Thanks for the update. As I mentioned, I think I figured it out with gsynth::gsynth alone so any additions would just be for convenience. Thanks again! M

feliuserra commented 4 years ago

Hi,

Sorry to "reopen" this and bother about this again, but I am having the same issue. When running the cumuEff function I get the following error:

Error in cumuEff(out, cumu = TRUE) : object 'D.boot' not found

@michaeldgarber could you please further develop how eff.boot allows you to compute yourself the cumulative effect? eff.boot shows a matrix with 3 dimensions: the first is the number of time periods, the second is the number of treated units and the third is the number of unique ids in the dataset. is that correct?

Perhaps this should go in another issue, but I will drop it here and if needed I can open a new issue.

Thank you very much in advance, Feliu

michaeldgarber commented 4 years ago

Hi Feliu, Sure, no problem. I will take a look this weekend and remind myself how I did it.

Best, Michael

On Thu, Apr 9, 2020 at 10:46 AM Feliu Serra Burriel < notifications@github.com> wrote:

Hi,

Sorry to "reopen" this and bother about this again, but I am having the same issue. When running the cumuEff function I get the following error:

Error in cumuEff(out, cumu = TRUE) : object 'D.boot' not found

@michaeldgarber https://github.com/michaeldgarber could you please further develop how eff.boot allows you to compute yourself the cumulative effect? eff.boot shows a matrix with 3 dimensions: the first is the number of time periods, the second is the number of treated units and the third is the number of unique ids in the dataset. is that correct?

Perhaps this should go in another issue, but I will drop it here and if needed I can open a new issue.

Thank you very much in advance, Feliu

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/xuyiqing/gsynth/issues/30#issuecomment-611567915, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCMIOC62B7JVY3QKIE2RP3RLXNTBANCNFSM4I5GOY3Q .

feliuserra commented 4 years ago

Hi Michael,

Thanks so much for the quick reply! Let me know if you need help or anything. Here is my email just in case: feliu.serra@bsc.es

Thanks again in advance, Best, Feliu

xuyiqing commented 4 years ago

Thank you, all. We'll look into it! It has been pretty hectic on our end.

On Thu, Apr 9, 2020 at 8:31 AM Feliu Serra Burriel notifications@github.com wrote:

Hi Michael,

Thanks so much for the quick reply! Let me know if you need help or anything. Here is my email just in case: feliu.serra@bsc.es

Thanks again in advance, Best, Feliu

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/xuyiqing/gsynth/issues/30#issuecomment-611592560, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB2PKGFYQ263JTANIK7IPYDRLXS37ANCNFSM4I5GOY3Q .

-- Yiqing Xu

Assistant Professor Department of Political Science Stanford University http://yiqingxu.org/

michaeldgarber commented 4 years ago

Sorry - I didn't get to this but hopefully mid week.

On Thu, Apr 9, 2020 at 11:31 AM Feliu Serra Burriel < notifications@github.com> wrote:

Hi Michael,

Thanks so much for the quick reply! Let me know if you need help or anything. Here is my email just in case: feliu.serra@bsc.es

Thanks again in advance, Best, Feliu

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/xuyiqing/gsynth/issues/30#issuecomment-611592560, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCMIOFUGSQ3SZOCIKHJ2WDRLXS37ANCNFSM4I5GOY3Q .

gl-smith commented 4 years ago

Hi @xuyiqing and @liulch ,

I have been having the same error where running cumuEff provides the error: Error in cumuEff(out, cumu = TRUE, id = NULL, period = c(0, 5)) : object 'D.boot' not found that others have mentioned.

For a point of reference, I am currently using the latest version of gsynth available on github (Version 1.1.6)

After looking at the source code, I noticed that D.boot first appears on line 61 in the line (if (sum(c(D.boot)) == 0)). It looks like D.boot is never defined prior to line 61 and thus the source of the error. Would you be able to share the purpose of the D.boot object? If this is the only problem, the bug might have a relatively easy fix.

Thanks for all of the hard work on this package! It's a great resource.

Best,

Greg

liulch commented 4 years ago

I can check it this weekend!

On Apr 14, 2020, at 1:52 PM, Gregory Smith notifications@github.com<mailto:notifications@github.com> wrote:

Hi @xuyiqinghttps://github.com/xuyiqing and @liulchhttps://github.com/liulch ,

I have been having the same error where running cumuEff provides the warning: Error in cumuEff(out, cumu = TRUE, id = NULL, period = c(0, 5)) : object 'D.boot' not found that others have mentioned.

For a point of reference, I am currently using the latest version of gsynth available on github (Version 1.1.6)

After looking at the source code, I noticed that D.boot first appears on line 61 in the line (if (sum(c(D.boot)) == 0)). It looks like D.boot is never defined prior to line 61 and thus the source of the error. Would you be able to share the purpose of the D.boot object? If this is the only problem, the bug might have a relatively easy fix.

Thanks for all of the hard work on this package! It's a great resource.

Best,

Greg

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/xuyiqing/gsynth/issues/30#issuecomment-613587676, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHGIZUFUYSLNHEGGIMDRQFDRMSPHVANCNFSM4I5GOY3Q.

xuyiqing commented 4 years ago

Thank you!

On Tue, Apr 14, 2020 at 10:38 PM Licheng Liu notifications@github.com wrote:

I can check it this weekend!

On Apr 14, 2020, at 1:52 PM, Gregory Smith <notifications@github.com mailto:notifications@github.com> wrote:

Hi @xuyiqinghttps://github.com/xuyiqing and @liulch< https://github.com/liulch> ,

I have been having the same error where running cumuEff provides the warning: Error in cumuEff(out, cumu = TRUE, id = NULL, period = c(0, 5)) : object 'D.boot' not found that others have mentioned.

For a point of reference, I am currently using the latest version of gsynth available on github (Version 1.1.6)

After looking at the source code, I noticed that D.boot first appears on line 61 in the line (if (sum(c(D.boot)) == 0)). It looks like D.boot is never defined prior to line 61 and thus the source of the error. Would you be able to share the purpose of the D.boot object? If this is the only problem, the bug might have a relatively easy fix.

Thanks for all of the hard work on this package! It's a great resource.

Best,

Greg

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub< https://github.com/xuyiqing/gsynth/issues/30#issuecomment-613587676>, or unsubscribe< https://github.com/notifications/unsubscribe-auth/AHGIZUFUYSLNHEGGIMDRQFDRMSPHVANCNFSM4I5GOY3Q>.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/xuyiqing/gsynth/issues/30#issuecomment-613827607, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB2PKGCIJP4XM2LTOKUTKXTRMVB6FANCNFSM4I5GOY3Q .

-- Yiqing Xu

Assistant Professor Department of Political Science Stanford University http://yiqingxu.org/

liulch commented 4 years ago

Hi all,

I fix the bug in the cumulative effect function and update it on GitHub. Seems that the name of the object D.boot should be Dtr.boot. I test it with the turnout data and find it works. Hope it will help!

Thank you, Licheng

On Apr 15, 2020, at 1:42 AM, Yiqing Xu notifications@github.com<mailto:notifications@github.com> wrote:

Thank you!

On Tue, Apr 14, 2020 at 10:38 PM Licheng Liu notifications@github.com<mailto:notifications@github.com> wrote:

I can check it this weekend!

On Apr 14, 2020, at 1:52 PM, Gregory Smith notifications@github.com<mailto:notifications@github.com mailto:notifications@github.com> wrote:

Hi @xuyiqinghttps://github.com/xuyiqing and @liulch< https://github.com/liulch> ,

I have been having the same error where running cumuEff provides the warning: Error in cumuEff(out, cumu = TRUE, id = NULL, period = c(0, 5)) : object 'D.boot' not found that others have mentioned.

For a point of reference, I am currently using the latest version of gsynth available on github (Version 1.1.6)

After looking at the source code, I noticed that D.boot first appears on line 61 in the line (if (sum(c(D.boot)) == 0)). It looks like D.boot is never defined prior to line 61 and thus the source of the error. Would you be able to share the purpose of the D.boot object? If this is the only problem, the bug might have a relatively easy fix.

Thanks for all of the hard work on this package! It's a great resource.

Best,

Greg

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub< https://github.com/xuyiqing/gsynth/issues/30#issuecomment-613587676>, or unsubscribe< https://github.com/notifications/unsubscribe-auth/AHGIZUFUYSLNHEGGIMDRQFDRMSPHVANCNFSM4I5GOY3Q>.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/xuyiqing/gsynth/issues/30#issuecomment-613827607, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB2PKGCIJP4XM2LTOKUTKXTRMVB6FANCNFSM4I5GOY3Q .

-- Yiqing Xu

Assistant Professor Department of Political Science Stanford University http://yiqingxu.org/

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/xuyiqing/gsynth/issues/30#issuecomment-613828824, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHGIZUARIBSNYH4OPLJ6NZ3RMVCODANCNFSM4I5GOY3Q.

feliuserra commented 4 years ago

Hi all,

Thanks to everyone for all the help. I have updated the gsynth package using devtools::install_github('xuyiqing/gsynth'). After that, the missing D.boot error no longer shows up. So, this error might be solved. Now it seems that D.boot and Dtr.boot are the same.

Also, I have checked the fix using the above mentioned code:

https://github.com/xuyiqing/gsynth/issues/30#issuecomment-539583648

And it works fine.

However, I have found a new error. I should probably write it in a new issue, but I'll leave it in here just in case. I am just going to leave the error here and open a new issue if you think it's appropriate.

I have an out output, obtained running:

 system.time(out <- gsynth(f, 
                              data = final_df,
                              index = c('ID_new', 'time'),
                              force = "two-way", 
                              r = 2,
                              se = TRUE, 
                              CV=1, 
                              lambda = .01,
                              inference='jackknife',
                              nboots = 10, 
                              parallel = TRUE, 
                              min.T0=125))

Where f is the formula used, final_df is the dataframe used, and the rest of the parameters are just regular parameters. Please contact me on my email in case you need the object itself or the dataframe used to create this out object. The number of unique time periods is 757, the number of treated units is 308, and the number of control units is 543.

So far, the code runs smoothly, and I have tried it with different parameters/estimators and it works well. However, when I intend to execute the cumuEff function:

cumuEff(out, cumu=TRUE, period=NULL, id=NULL)

I obtain a new error. The error looks as follows:

Error in ts:te : result would be too long a vector

Googling for a similar error, I found that this might be because of the size of the panels that I am using. I am using a MacOS with 8 cores, i9 and 64GBs of RAM.

Then, I have tried tweaking the parameters available, such as period, and I still get the same error.

Can anyone point me in the right direction for the solution of this?

Sorry for the long post and thanks again. If someone believes that this should be a different issue please let me know and I will create it.

Thanks again for everything,

Best,

Feliu

xuyiqing commented 4 years ago

Could you send us a sample of your dataset and code such that we can replicate this error?

yiqingxu@stanford.edu; liulch@mit.edu

On Wed, Apr 22, 2020 at 9:46 AM Feliu Serra Burriel < notifications@github.com> wrote:

Hi all,

Thanks to everyone for all the help. I have updated the gsynth package using devtools::install_github('xuyiqing/gsynth'). After that, the missing D.boot error no longer shows up. So, this error might be solved. Now it seems that D.boot and Dtr.boot are the same.

Also, I have checked the fix using the above mentioned code:

30 (comment)

https://github.com/xuyiqing/gsynth/issues/30#issuecomment-539583648

And it works fine.

However, I have found a new error. I should probably write it in a new issue, but I'll leave it in here just in case. I am just going to leave the error here and open a new issue if you think it's appropriate.

I have an out output, obtained running:

system.time(out <- gsynth(f, data = final_df, index = c('ID_new', 'time'), force = "two-way", r = 2, se = TRUE, CV=1, lambda = .01, inference='jackknife', nboots = 10, parallel = TRUE, min.T0=125))

Where f is the formula used, final_df is the dataframe used, and the rest of the parameters are just regular parameters. Please contact me on my email in case you need the object itself or the dataframe used to create this out object. The number of unique time periods is 757, the number of treated units is 308, and the number of control units is 543.

So far, the code runs smoothly, and I have tried it with different parameters/estimators and it works well. However, when I intend to execute the cumuEff function:

cumuEff(out, cumu=TRUE, period=NULL, id=NULL)

I obtain a new error. The error looks as follows:

Error in ts:te : result would be too long a vector

Googling for a similar error, I found that this might be because of the size of the panels that I am using. I am using a MacOS with 8 cores, i9 and 64GBs of RAM.

Then, I have tried tweaking the parameters available, such as period, and I still get the same error.

Can anyone point me in the right direction for the solution of this?

Sorry for the long post and thanks again. If someone believes that this should be a different issue please let me know and I will create it.

Thanks again for everything,

Best,

Feliu

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/xuyiqing/gsynth/issues/30#issuecomment-617897201, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB2PKGB6TN2PWP7VY37F2LLRN4NPNANCNFSM4I5GOY3Q .

-- Yiqing Xu

Assistant Professor Department of Political Science Stanford University http://yiqingxu.org/

feliuserra commented 4 years ago

Hi,

So, after giving it one more thought, I tried with a parametric estimator, instead of jackknife, and it worked. So either, this error is related to the 'jackknife' and cumulative effect for large samples, because it runs into a memory error, or something else that I am missing.

Thanks again for your help and interest, I believe there is no need to bother you anymore.The D.boot error has been solved.

Feliu