IMTEK-Simulation / NuMPI

Utilities for MPI-parallel numerical calculations with Python
MIT License
2 stars 1 forks source link

NCStructuredGrid stores NaNs on 32768x32768 fields and/or large number of cores (320) #53

Open sannant opened 3 years ago

sannant commented 3 years ago

Additionally, a boolean field (boolean in python, int64 once in netCDF) possesses -1 values.

I was not able to coin down what is going on. The same simulation setup works fine on 8192x8192 grid and 20 cores

@pastewka proposed that it could be a type conversion issue, that in the large simulations large floats appear that turn into NaN in some light-weight float types.

ncdump -h shows it is a "double" fields.

However, that would not explain the -1 in the boolean field.

For an example of such a simulation, see the dataset 543d22d6-c57d-4b3d-b8d3-895d2dc51af6. The field pressures contains NaN in the NetCDF file but is fine in the .npy export.

Because the installation of the MPI dependencies of NCStructuredGrid is a mess and we do not want to maintain two NetCDF wrappers, we will not seek the problem here and use the muSpectre wrapper instead in future. NCStructuredGrid is still usefull in serial and/or as legacy support for old simulation scripts.