search

evanoconnor / NuLib

open-source neutrino interaction library
23 stars 16 forks source link

openMP and electron capture rates #14

Open evanoconnor opened 8 years ago

evanoconnor commented 8 years ago

I've seen issues here, but only with some compilers. Problems seems to arise in Hempel EOS routine which is likely not thread safe. MPI works fine, as does 1 thread.

csullivan commented 8 years ago

Do you have an idea of which compiler versions work / do not work?

One thing I ran into when working on the openMP component was that it did not play well with derived types (essentially f90 structs) which contain allocatable arrays. Specifically, openMP cannot thread private derived types with allocatable arrays (this is discussed some in the comments here: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=38724). My solution was to use pointers and manually allocate the needed data, and I can imagine the implementation of this being compiler specific.

If it's unrelated to that, then I would agree the first place to look is probably the Hempel EOS routine

csullivan commented 7 years ago

Hi Evan, just noticed this is still open. Did this ever get resolved?

evanoconnor commented 7 years ago

No it hasn't, Since I can avoid it by just using the MPI version I haven't had to spend the time to further explore it.

csullivan commented 7 years ago

Ok, if you are able to provide access to the machine your running on, I'd be happy to look into it. I use the openmp version pretty frequently. At least it would be good to know whether or not my assumption about it having to do with the compiler version is correct.

csullivan commented 7 years ago

I think I found the issue. Try adding

!$OMP THREADPRIVATE(coul,degen)

around line 86 of sfho_frdm_composition_module.f

This fixed an issue with openmp nulib someone was having here. If this does work, then I can update the SFHo section of the README.

evanoconnor commented 7 years ago

I'll try this again, but I have done it in the past and I recall the issue still existed (I have that exact line in my sfho_frdm_composition_module.f).

Regardless, I think for openmp this is a crucial statement to have as otherwise those variables are shared among threads.