gvegayon / parallel

PARALLEL: Stata module for parallel computing
https://rawgit.com/gvegayon/parallel/master/ado/parallel.html
MIT License
118 stars 26 forks source link

When there are nested loops, which loop is executed in parallel? #112

Closed yanyu2015 closed 1 month ago

yanyu2015 commented 1 month ago

When I have two loops in my defined ‘prog’ command, for example:

capture program drop parforreg
prog def parforreg
    use "1.dta", clear
    global Xs “var1 var2 var3”
    forvalues i = 1/100 {
        foreach Xi of global Xs{
            do something
        }
    }
end

or

capture program drop parforreg
prog def parforreg
    use "1.dta", clear
    global Xs “var1 var2 var3”
    foreach Xi of global Xs{
        forvalues i = 1/100{
            do something
        }
    }
end

When I run the following commands to parallel

parallel setclusters 4
parallel, prog(parforreg) nodata: parforreg

Is it the outer loop that is executed in parallel, or the inner loop that is executed in parallel, or is there a different strategy for foreach versus forvalues?

bquistorff commented 1 month ago

parallel does not parallelize a generic loop. You'll likely have to rewrite your code to use one of the existing parallelization strategies. The help lists the variety of approaches.

yanyu2015 commented 1 month ago

I have updated my question to better describe my problem. My approach is based on the example of parfor you provided. I believe I am using the strategy of 'Parallelizing a Stata command'. However, there don't seem to be more detailed examples to illustrate the specifics of this strategy, so I am currently confused about how this strategy parallelizes custom commands. Or could you suggest which strategy in the help documentation I should look at to achieve the above functionality? Are there more examples for reference? I also don't know what approach I should take to rewrite the code. I hope you can guide me. Thank you very much!

yanyu2015 commented 1 month ago

I review the intermediate output of this command, and I already know where the issue lies.