Closed kserradell closed 8 months ago
One big factor to parallel I/O performance is the file system.
One possibility is /gxfs/scratch/process/kserradell
is a local file system
on mmlogin01, but a remotely mounted file system on mmnode001.
Command mount
and df
can reveal that information.
You can ask the system administrator for the file system on your HPC machine that should be used for parallel I/O.
Hello @wkliao
Thanks for your answer. No, in both cases we are writing in GPFS.
The main difference was that the login node has HyperThreading activated and not in compute nodes. We noticed if you use the full node with MPI tasks for the benchmark, there is not enough "room" for other OS tasks, impacting the overall performance of the benchmark. Activating HT and using 128 in a 256 cores node, solved the issue.
It seems the problem is resolved. This issue can be re-opened if needed.
Hello
We are deploying a new HPC and we are facing a performance issue. PnetCDF performs worse in applications when we execute in the compute nodes than in the login nodes. We have been doing many tests and changes but we are still clueless about the cause. Both nodes are hardware identical (also NUMA distribution) and the main change is OS (RedHat 8.6) vs. (Rocky Linux 8.6) but should be really similar.
Using IOR and NCMPI driver, we don't appreciate those differences. Only in our main app (WRF) and in the FLASH-IO benchmark.
Any help and/or indications to look at would be highly appreciated. I can provide more details if needed.
Thanks,
KiM