Screen Diagnostic Not Working

Nkehoe-QUB commented 3 months ago

Description

I'm trying to add a screen diagnostic in to my simulation but I get the following error output: TempDist.txt

It seems to be when I try to use multiple nodes, it fails. Is this the issue and is there a work around? It's not viable to run the simulation on a single node.

If available, copy-paste faulty code, warnings, errors, etc.

Steps to reproduce the problem

If relevant, provide a step-by-step guide

Parameters

Smilei version ?
Input file ?
Commands or script used to run the faulty simulation ?
Computing resources
- type of computer
- OS
- C++ compiler (to get some information g++ --version)
- MPI version (mpic++ --version, mpic++ -show)
- HDF5 version (h5cc --version)
- Python version (python --version)
- ...

mccoys commented 3 months ago

Maybe not enough memory for MPI. Make sure that your diagnostic size is reasonable

Nkehoe-QUB commented 3 months ago

When I run on a single node (128 cores, 128GB RAM) it will start okay but if I try try 2 nodes (256 cores, 256GB RAM) it quits at this step:

 Running diags at time t = 0
 -------------------------------------------------------------------------------

I don't think it's a memory issue as its only initialising the diagnostic.

mccoys commented 3 months ago

It works on 1 node because then MPI communication is not needed. It fails for communication likely due to limited memory for MPI.

The diag is initializing and writing. There is definitely a communication

Nkehoe-QUB commented 3 months ago

Okay, I'll try increasing the amount of memory and see if that helps.

Nkehoe-QUB commented 3 months ago

Okay, you were correct. I increased the memory and it seems to be running okay now! I appreciate the help!

SmileiPIC / Smilei