Making additions to Depends.nvhpc to enable GPU compilation of clubb code.

ESMCI / ccs_config_cesm

CESM CIME Case Control System configuration files

3 stars 46 forks source link

Making additions to Depends.nvhpc to enable GPU compilation of clubb code. #171

Closed huebleruwm closed 5 months ago

huebleruwm commented 5 months ago

These changes were tested on Derecho with

cam tag: cam6_3_155
clubb tag: clubb_4ncar_20240605_73d60f6 (https://github.com/larson-group/clubb_release/releases/tag/clubb_4ncar_20240605_73d60f6)
ccs_config tag: ccs_config_cesm0.0.106
nvhpc version: 24.3
resolution: ne30pg3_ne30pg3_mg17
compset: F2000dev
addition parameters: --ngpus-per-node 4 --gpu-type a100 --gpu-offload openacc

sjsprecious commented 5 months ago

@jedwards4b Gunther asked a question saying:

Out of curiosity, do you know if there's a way to make this list of files more general? E.g applying these flags to all the files in /src/physics/clubb/src/CLUBB_core without having to list them individually. It would be nice to not have to edit this file in the future if another CLUBB file is added that we would want GPU support for.

I think I asked you a similar question long time ago and the answer was no at that moment. Is it possible to apply GPU flags to a whole folder now in the latest CIME? Thanks.

jedwards4b commented 5 months ago

I think that you could do something in the cam buildlib script to do that. The Makefile targets are keyed from file extension in general

.F90.o:
    $(FC) -c $(INCLDIR) $(INCS) $(FFLAGS) $(FREEFLAGS) $(CONTIGUOUS_FLAG) $<

You could try modifying the filename extension - maybe instead of .F90 they are .G90, this would allow you to do

 .G90.o:
      $(FC) -c $(INCLDIR) $(INCS) $(FFLAGS) $(FREEFLAGS) $(GPUFLAGS) $<

there are a number of ways to do what you want to do. You'll have to explore a bit to come up with the best solution.

sjsprecious commented 5 months ago

Thanks @jedwards4b for your suggestions. That is good to know.