Change Makefile.common so that 'make new' does compilations in parallel

[rjleveque]

First the module files have to be compiled (in serial) and then all the other Fortran files can be compiled in parallel with the '-j' flag.

This only does parallel compile when doing 'make new'. If doing 'make .exe' then only Fortran files that have changed need to be recompiled and there's usually only a few. It's not clear how one would do these in parallel since each compilation is triggered separately when the .o file is checked.

[bolliger32]

@rjleveque @mandli I'm now getting an error when trying to compile with make new and realizing this might be tied to this recent PR? It seems to not be able to recognize the rules for building all the .mod files and I get an error like the following. Does this ring a bell to you? Happy to dig in a bit further if not.

...
make     /home/jovyan/git-repos/arhe.csms.clawpack/amrclaw/src/2d/amr_module.mod /home/jovyan/git-repos/arhe.csms.clawpack/amrclaw/src/2d/regions_module.mod /home/jovyan/git-repos/arhe.csms.clawpack/amrclaw/src/2d/adjoint_module.mod /home/jovyan/git-repos/arhe.csms.clawpack/geoclaw/src/2d/shallow/utility_module.mod /home/jovyan/git-repos/arhe.csms.clawpack/geoclaw/src/2d/shallow/geoclaw_module.mod /home/jovyan/git-repos/arhe.csms.clawpack/geoclaw/src/2d/shallow/topo_module.mod /home/jovyan/git-repos/arhe.csms.clawpack/geoclaw/src/2d/shallow/qinit_module.mod /home/jovyan/git-repos/arhe.csms.clawpack/geoclaw/src/2d/shallow/refinement_module.mod /home/jovyan/git-repos/arhe.csms.clawpack/geoclaw/src/2d/shallow/fgout_module.mod /home/jovyan/git-repos/arhe.csms.clawpack/geoclaw/src/2d/shallow/fgmax_module.mod /home/jovyan/git-repos/arhe.csms.clawpack/geoclaw/src/2d/shallow/surge/model_storm_module.mod /home/jovyan/git-repos/arhe.csms.clawpack/geoclaw/src/2d/shallow/surge/data_storm_module.mod /home/jovyan/git-repos/arhe.csms.clawpack/geoclaw/src/2d/shallow/surge/storm_module.mod /home/jovyan/git-repos/arhe.csms.clawpack/geoclaw/src/2d/shallow/gauges_module.mod /home/jovyan/git-repos/arhe.csms.clawpack/geoclaw/src/2d/shallow/multilayer/multilayer_module.mod /home/jovyan/git-repos/arhe.csms.clawpack/geoclaw/src/2d/shallow/friction_module.mod /home/jovyan/git-repos/arhe.csms.clawpack/geoclaw/src/2d/shallow/adjointsup_module.mod  # also makes MODULE_OBJECTS
make[1]: Entering directory '/home/jovyan/git-repos/arhe.csms.clawpack'
make[1]: *** No rule to make target '/home/jovyan/git-repos/arhe.csms.clawpack/amrclaw/src/2d/amr_module.mod'.  Stop.
make[1]: Leaving directory '/home/jovyan/git-repos/arhe.csms.clawpack'
make: *** [/home/jovyan/git-repos/arhe.csms.clawpack/clawutil/src/Makefile.common:281: new] Error 2

[rjleveque]

@bolliger32: yes it could be related. We were running into problems with gauges_module.mod being picked up from amrclaw/src/2d rather than from geoclaw and tried to fix things up but might have broken something.

Strange that it says it can't make amr_module.mod. I just did some more tests with the latest and even if I remove amrclaw/src/2d/amr_module.mod it gets remade when I do make .exe after changing a file that uses it in geoclaw.

Can you confirm that things still work ok for you if you go back to the previous commit in geoclaw before this merge?

[mandli]

I am working on looking at these changes and testing to see what seems to have changed to make some things break. Will update here when I have a better view of what's going on.

[bolliger32]

Hi @mandli and @rjleveque - sorry this kind of slipped as I was wrapping up work and over the holidays. But I just tried again with the current HEAD of the master branch and I no longer have the issue, so it seems like whatever fixes you made seem to have worked! Let me know if that’s surprising and I can look into things further, but otherwise I’ll consider this solved! Thanks!

[rjleveque]

@bolliger32: Thanks for testing, I also think it's working fine now along with a few other recent changes to the Makefile, but let us know if you run into other problems with it.

[bolliger32]

Hi @rjleveque and @mandli - It seems this issue is back again. I'm not sure if it was ever solved or if I might have forgotten to specify "new" as my make target when I tested on Dec 30. Right now, I can confirm that make .exe works fine but make new runs into that same error where it can't seem to understand that there's a rule to make all %.mod files. I still get:

make[1]: *** No rule to make target '/srv/clawpack/amrclaw/src/2d/amr_module.mod'.  Stop.

I'm thinking this maybe has something to do with the recursive make call within the new target? I'm currently running with the master branch from all submodules. Are you guys able to make new without this error?

[bolliger32]

oh! its because my original make call was using a file called Makefile.clawpack so it required a -f Makefile.clawpack specification. The recursive make calls within the new target don't invoke any -f flag and are probably executing back from the original directory where I called make from. So it doesn't know to use Makefile.clawpack and thus doesn't know to include Makefile.common and thus doesn't see the rules to build %.mod. I'm not super familiar with makefiles. Would the right way to solve this be to somehow pass any flags from the original call to these recursive calls?

[rjleveque]

@bolliger32: Could you see if #170 fixes your problem?

[bolliger32]

@bolliger32: Could you see if #170 fixes your problem?

@rjleveque It does! thanks!

[rjleveque]

Great, thanks Ian! And it also fixes a similar problem I ran into with @mjberger.