search

sergiocorreia / reghdfe

Linear, IV and GMM Regressions With Any Number of Fixed Effects
http://scorreia.com/software/reghdfe/
MIT License
219 stars 57 forks source link

[BUG] xtabond2 problems #216

Closed asmithbravo closed 4 years ago

asmithbravo commented 4 years ago

Dear Sergio,

After running xtabond2 for a dynamic panel data model I get the same message:

round(): 3900 unable to allocate real [403200000,1] factor(): - function returned error

: - function returned error Could I use tools to estimate xtabond2 and solve the memory problem When I add the nomata option, the system tells me the following message: no room to add more variables Up to 5,000 variables are currently allowed, although you could reset the maximum using set maxvar; see help memory.] Do you know how to solve this_ Could I use your package? Thanks
sergiocorreia commented 4 years ago

This looks like a standard out-of-memory problem. Stata tried to allocate a huge vector with 400m rows and failed.

asmithbravo commented 4 years ago

Dear Sergio,

Thanks for your reply, but I meant my case, not the 400 million rows. This is why I don't understand why does it happened to me. I have a panel from 1960 to 2019, and I am using around 6 gmm style variables (including the lagged dependent variable) and 1 ivstyle (year fixed effects), and still see the same error message.

sergiocorreia commented 4 years ago

I'm not fully sure of how xtabond2 uses ftools or reghdfe though. I went through the source (viewsource xtabond2.mata) and found no mention of either.

asmithbravo commented 4 years ago

Thanks. Are you aware of how to fix this memory problem with xtabond2?

droodman commented 4 years ago

xtabond2 does not use reghdfe and Sergio did not write xtabond2, so this question would seem not to be for him.

That said, xtabond2 does not call round() or factor(), so I don't think that error message is coming from xtabond2.

You need to document more precisely and fully what you are doing and what problems result and determine whether the discussion even belongs here.

If you are running out of memory in xtabond2, it's probably because you are generating a lot of instruments, which is usually a problem. See my paper, "A Note on the Theme of Too Many Instruments."

asmithbravo commented 4 years ago

Dear David,

Thanks for your reply and for the very good paper suggested. Here is the link where I posted my question.
https://github.com/bashtage/linearmodels/issues/95#issuecomment-727165190