search

MPAS-Dev / MPAS

Repository for private MPAS development prior to the MPAS v6.0 release.
Other
4 stars 0 forks source link

Add optimization flag to fix irreproducibility on KNL/Intel #1520

Closed philipwjones closed 6 years ago

philipwjones commented 6 years ago

Added the -fp-model consistent flag to Intel compiler optimization options to fix irreproducible results on some architectures (namely KNL) Also, replace deprecated -FR flag with -free for free form source using Intel

Tested on Cori-KNL and Edison

mgduda commented 6 years ago

I think it would be better to target the develop branch with this PR, and to add the framework label (and to name the branch with the framework/ prefix rather than ocean/. My concern is that if infrastructure changes are brought in through a core-specific development branch, that branch may get merged to develop without any other cores noticing the infrastructure changes.

@philipwjones What sort of performance impact (if any) are you seeing when adding the -fp-model consistent flag? In the past, I think we found a small performance degradation in the atmosphere core when adding -fp-model precise, so we decided to not add this flag by default.

mgduda commented 6 years ago

With the Intel 16.0.2 compilers, I'm getting the following warning when compiling the branch in this PR:

ifort: command line warning #10159: invalid argument for option '-fp-model'

Would it be better to consider using -align array64byte?

philipwjones commented 6 years ago

@mgduda : -align array64byte certainly fixes the issue and should minimize other performance impacts. There's apparently another flag for Intel 18 that's more similar to the -fp-model flag but with fewer side effects too, but that wouldn't work for Intel 16, 17. I just chose -fp-model consistent since it was the safest option in case other non-reproducible optimizations bite us later, but it sounds like the -align array64byte might be best in the short term if it's available in Intel16 And sorry - I forgot this was more of a framework thang...so if we need to redo with the align option, I can rename at that point. Leaving Sunday and will be largely out of contact for a week, so might not get to this again for a week or so.

philipwjones commented 6 years ago

Replaced by MPAS-Dev/MPAS-Model#37 that addresses issue above and reports in new repo