search

davidkleiven / CEMC

DEPRECATED: Monte Carlo package targeted at systems studied with the Cluster Expansion.
MIT License
3 stars 2 forks source link

parallel computing error #51

Closed phymalidoust closed 5 years ago

phymalidoust commented 5 years ago

When running on a supercomputer, with 1node and asking for 1 task per node, it works although returns several warnings. However, when the number of nodes or tasks per node is increased, the code stops with error. The output files of both cases are attached.

output_1node_1task.txt output_3nodes_1task.txt

davidkleiven commented 5 years ago

I have a few questions

  1. What was the exact command used to install the package?
  2. Which modules where loaded (output of module list)
  3. Which compilers where used (I guess it is Intel or GCC)
davidkleiven commented 5 years ago

How are you parallelizing? It looks like these are extremely small calculations with only 2500 mc steps. How long does each take, a few seconds?

phymalidoust commented 5 years ago

questions: 1- pip3 install -e . --user (på idun) 2,3- gcc-8, mpi, python3 ... (it does not complain about missing compiler. I'm getting results with 1node-1task.). Also, I have installed and load the compiler/module from my own directory

I assumed it is internally parallelized and I don't need to make the outer for-loop parallelized. Am I correct? Please note that it raises these errors even when 1node and 1 task per node is requested. It complains about something inside the MC code.

The steps' time sounds reasonable but didn't check it out how long they take exactly.

davidkleiven commented 5 years ago

Monte Carlo sampling is a Markov Chain so you cannot parallelize that. There are a couple of options

  1. If you use a Markov Chain to sample the configurational space the only thing you can do is to start multiple Markov Chains and average the outcome of them.
  2. If you search for ground state you can start multiple searches and take the one having lowest energy as the ground state
  3. If you have multiple parameters (like several compositions) you can parallelize over that

For your case I think option 3 is the best. Which means that you have to parallelize a for loop over compositions.

davidkleiven commented 5 years ago

Option 1 is handled internally, but you have to pass a MPI communicator object to the Monte Carlo class to activate it. For ground search I recommend option 3.

phymalidoust commented 5 years ago

OK, understand. Will try to make an example of option 3 and then maybe you could complement it with option 1.

davidkleiven commented 5 years ago

Option 1 is of no use when searching for ground states. At low temperatures it might also be rather inefficient because all Markov Chain can easily end up visiting the same states and then there is not much point. If you really want to make sure that you use the CPUs efficiently option 3 is the best.

Den lør. 20. okt. 2018, 10.54 skrev phymalidoust notifications@github.com:

OK, understand. Will try to make an example of option 3 and then maybe you could complement it with option 1.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/davidkleiven/CEMC/issues/51#issuecomment-431562394, or mute the thread https://github.com/notifications/unsubscribe-auth/AMg_T00qqBEsfEw_O1u-cv6hZBBEgky1ks5umuTCgaJpZM4XwNJx .