Limited multi-processor use?

[Danferno]

You've probably had this question before (although I didn't find it), but are there any plans to make hdfe/reghdfe use more processors simultanously? At the moment it seems limited to 1-4 processors even on a Stata-MP64 license.

[sergiocorreia]

Stata provides no MP primitives, so we can only exploit the fact that some Mata functions already support multiprocessing. So if there is a limit it's probably there, and not on ftools/reghdfe side.

Beyond that, I tried to implement reghdfe via opening multiple stata processes, which is not super fast but better than nothing. However, when I did so (roughly a year ago) I found some Mata errors that didn't allow me to open Mata objects (Stata just crashed). Not sure if they are fixed or not, but haven't seen anything in the recent "whatsnew" of stata

[Danferno]

Ah that's unfortunate, thanks for the response though (and for reghdfe of course)!

[sergiocorreia]

It's definitely unfortunate, and up high in my to-do list whenever doing this becomes feasible!

[meadover]

Sergio, when you discover bugs, do you report them to Stata? When I found a bug in their graphics package a month ago, the developers fixed it within a few days in time for the next release.

Mead

From: Sergio Correia notifications@github.com Sent: Tuesday, December 8, 2020 12:04 PM To: sergiocorreia/reghdfe reghdfe@noreply.github.com Cc: Subscribed subscribed@noreply.github.com Subject: Re: [sergiocorreia/reghdfe] Limited multi-processor use? (#218)

It's definitely unfortunate, and up high in my to-do list whenever doing this becomes feasible!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/sergiocorreia/reghdfe/issues/218#issuecomment-740765863, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ALEY4Q3HRQCT7WJLBOJD7C3STZMBXANCNFSM4URXGHRA.

[sergiocorreia]

That's usually been my case with the easy bugs, but not with the trickier ones. Interestingly, the email was sent exactly a year ago, on 08Dec2019.

To reproduce the bug, just copy-paste-run this snippet:

clear all
cls

mata:

class MyClass
{
    void new()
    transmorphic scalar myarray
}

void MyClass::new()
{
    myarray = asarray_create()
}

f = MyClass()
mata matsave "object.tmp" f, replace
mata matdesc "object.tmp"

end

mata: mata desc
mata: mata matuse "object.tmp", replace

exit

Be aware that your Stata will hard crash though.

Reply:

Dear Sergio,

Thank you for tracking down the trigger for this issue. I have passed it along
to our developers so they can fix this bug in a future update.

Sincerely,

Note that I haven't been able to test the latest versions because at the Fed we are always a bit behind on updates (due to security). However, I don't see anything about it on the changelog