Updating config site files require lots of manual changes

[JustinPrivitera]

To update the config site file for magma (#19364), I had to make lots of manual changes. It would be nice if when the config site file was generated, it could handle some of these things.

I needed to add back in

VISIT_OPTION_DEFAULT(VISIT_SLIVR TRUE TYPE BOOL)

##
## Turn on DDT support.
##
VISIT_OPTION_DEFAULT(VISIT_DDT ON TYPE BOOL)

I had to use the original Parallel Build Setup instead of the new one.

##
## Parallel Build Setup.
##
VISIT_OPTION_DEFAULT(VISIT_PARALLEL ON TYPE BOOL)
VISIT_OPTION_DEFAULT(VISIT_MPI_CXX_FLAGS -I/usr/tce/packages/mvapich2/mvapich2-2.3.6-gcc-10.3.1/include TYPE STRING)
VISIT_OPTION_DEFAULT(VISIT_MPI_C_FLAGS   -I/usr/tce/packages/mvapich2/mvapich2-2.3.6-gcc-10.3.1/include TYPE STRING)
VISIT_OPTION_DEFAULT(VISIT_MPI_LD_FLAGS  "-L/usr/tce/packages/mvapich2/mvapich2-2.3.6-gcc-10.3.1/lib -Wl,-rpath=/usr/tce/packages/mvapich2/mvapich2-2.3.6-gcc-10.3.1/lib" TYPE STRING)
VISIT_OPTION_DEFAULT(VISIT_MPI_LIBS     mpich mpl)
VISIT_OPTION_DEFAULT(VISIT_PARALLEL_RPATH  "/usr/tce/packages/mvapich2/mvapich2-2.3.6-gcc-10.3.1/lib")

The new one was a one liner setting VISIT_MPI_COMPILER

Those are the things I can recall now.

[cyrush]

@biagas will double check if we need slivr at all for VTK-9 builds

[biagas]

VISIT_SLIVR isn't strictly necessary to be in the config-file, as it is an option that is ON by default. https://github.com/visit-dav/visit/blob/c02b56f052d2cbafaeff3e8e1e8a9416ac31b3ca/src/CMake/VisItOptions.cmake#L12