Use pre-built tools when available

[amametjanov]

The tools (namelist_gen, streams_gen, parse) are always built and used on login nodes. When TOOL_TARGET_ARCH is different for login and compute nodes, care is needed to define the right TOOL_TARGET_ARCH for a given login node (sandybridge, haswell, istanbul, ...). Otherwise, errors like Illegal instruction are likely. It might be better to build a static version of the tools once and keep them in a shared directory. This PR copies over pre-built tools and avoids re-building them when available: e.g. make TOOL_DIR=$shared_dir ... .

[mark-petersen]

@amametjanov I just rebased your commit on the head of develop. Then I added a commit. In 39c6eba2a6acf8ab04fb8c787ef4faee0c17c171 I add subdirectories. I assume you want to point to a single pre-compiled tools directory. Now you can compile with something like this:

export TOOLS_DIR=/usr/projects/climate/mpeterse/repos/MPAS/ocean_develop/src/tools
make ifort CORE=ocean 

where that path is something you've compiled before.

I made a commit 2b24857389de01d43b230857921daf1982bedd7c to add lines to pass the TOOLS_DIR argument down to the tools makefile. But then I realized that we use export for PIO, PNETCDF, etc. So I took it off again.

Could you make sure the additional commit is workable for what you need?

[mark-petersen]

@mgduda and @matthewhoffman, can I merge this soon? It simply adds an optional environmental variable TOOLS_DIR path so that @amametjanov can point to a pre-compiled directory. If TOOLS_DIR is not defined it works the same as before.

[amametjanov]

Yes, I think the commit with sub-directories is good.

[mgduda]

I think it would be good to include some echo statements in the Makefile to indicate whether tools are being used from some other directory or not. My concern is that the build process is becoming opaque, with various environment variables controlling different options with no clear indication to the user what is actually being done. For example:

ifdef TOOL_DIR
        @echo "*** Using MPAS tools from ${TOOL_DIR} ***"
        (cp $(TOOL_DIR)/input_gen/namelist_gen input_gen/)
        (cp $(TOOL_DIR)/input_gen/streams_gen input_gen/)
else
        echo "*** Building MPAS tools from source ***"
        (cd input_gen; $(MAKE) CPPFLAGS="$(CPPFLAGS)" CC="$(CC)" CFLAGS="$(CFLAGS)")
endif

Also, what do you think about using MPAS_TOOL_DIR rather than TOOL_DIR? The latter is rather generic and could conceivably be used for other purposes on some systems. I know we already use TOOL_TARGET_ARCH, and in retrospect it may have been nice to have prefixed that with MPAS_ as well.

[amametjanov]

Updated the branch.

[mark-petersen]

Great. I'll merge it in a bit.