How to set amrex parameters from code?

[nncarlson]

Many of the procedures in the Fortran amrex interface are getting input parameters not from arguments, but from a hidden "parmparse" database. Is there a way for Fortran code to set parameters in that database? The issue is that we (ECP ExaAM) do not want to use the amrex input file format. We have our own system for managing input parameters, and want to use amrex as a library and not as the environment we operate in.

[zingale]

I don't think that this does exactly what you want, but perhaps you could modify it for your use.

In Castro, we list all of the runtime parameters in a plain text file:

https://github.com/AMReX-Astro/Castro/blob/master/Source/_cpp_parameters

there is an column there that specifies whether the parameter is needed in Fortran too.

We then build C++ header files and a Fortran module based on this list using the script:

https://github.com/AMReX-Astro/Castro/blob/development/Source/parse_castro_params.py

We do this manually, but if desired it could be done at compile time too by hooking it into the makefile.

At the moment, this only works for our application-specific parameters, not those in the amr namespace.

[nncarlson]

Thanks for the suggestion @zingale but it's not really what I'm looking for. We use hierarchical "parameter list" objects for disseminating parameters throughout our codes, and can populate them by reading JSON-format input files (among other ways). This is modeled after the ParameterList class from Trilinos' Teuchos package. I don't expect amrex to know anything about this, but what I do expect, and am looking for, is a way to "pass" parameters to amrex procedures without putting them into a separate amrex input file. I'm hoping there might be some calls that can set parameters in their database.

[WeiqunZhang]

AMReX initializes the database with arguments passed to amrex::Initialize.

For the Fortran interface, AMReX provides a main function in amrex/Src/F_Interfaces/Base/AMReX_fi_main.cpp. One can override the main function there. I believe that's probably what you do. In that case, you can call

amrex::Initialize(argc,argv);

with argc and argv provided by your code.

Alternative approach: In C++, we do have add functions that can be used to add parameters to the database after it is initialized. This has not bee ported to Fortran. We can certainly provide that as well.

[zingale]

alternately, we could write a Fortran version of pp.add() to add to the database.

[WeiqunZhang]

Forgot to mention, the arguments passed to amrex::Intitalize are expected to be

exectuable [inputs_file] [parm_1=value] [amrex.parm_4=value] [mynamesapce.parm_2=value]

[nncarlson]

Yes, it sounds like a Fortran version of pp.add() is exactly what I'm looking for.

For amrex::Initialize is it possible to omit the the inputs_file argument? Will this properly create an empty database, that can then be populated appropriately with pp.add() calls?

Incidently, it seems it would be possible to have a Fortran main that would call the requisite amrex startup and shutdown procedures. It would require me writing some thin Fortran-callable C wrappers around calls to the C++ methods. But I don't see any issues otherwise. Would you agree?

[WeiqunZhang]

We will add Fortran version of ParmParse::add.

Yes, the inputs file argument can be omitted. That's what is meant with [].

I don't see any issues. We will add wrappers to amrex::Initialize etc. so that the main can be written in Fortran.

[WeiqunZhang]

@nncarlson I think the latest development branch does what you want. Please take a look at Tutorials/Basic/main_F/ for an example.

[nncarlson]

I'm finally getting to trying this. What are the optional arg_parmparse and proc_parmparse arguments to amrex_init?

[WeiqunZhang]

Usually, we run with executable inputs_file parm_1=value_1 parm_2=value_2. Here inputs_file is a text file containing runtime parameters. One can also overwrite the parameters in the file in the command line with parm=value following the file name. Both the file and parm=value are optional. So there are four possibilities AMReX usually expects.

1. executable
2. executable inputs_file
3. executable parm=value
4. executable inputs_file parm=value

But this may not be what one wants. If your code is run as executable my_file_not_understood_by_amrex and AMReX fails to parse my_file_not_understood_by_amrex as ParmParse inputs file, this may result in a runtime error. The optional argument arg_parmparse (defaulted to true), if set to false, will make AMReX skip parsing any command line arguments.

Even though you may not want AMReX to parse the command line, you may still want to use ParmParse to add your own parameters or change AMReX's default parameters. For that, you can provide a procedure say subroutine add_parameters () bind(c) as proc_parmparse. That will be run inside amrex_init. The reason we have to pass a procedure instead of calling it after amrex_init is called is some AMReX parameters must be set inside amrex_init not after, if one wants to change their default values.

[nncarlson]

Excellent, arg_parmparse is exactly what I need.

Is there a list somewhere of those parameters that must set within amrex_init vs those that can be set after calling it? Sorry for the newbie questions.

[WeiqunZhang]

Unfortunately, we don't have such a list. So I would recommend setting all AMReX parameters in the proc_parmparse procedure.

Anyway, here is a list of some important parameters and their default values off the top of my mind:

• amrex.fpe_trap_invalid = 0 # trap nans or not
• amrex.fpe_trap_zero = 0 # trap divide by zero or not
• amrex.fpe_trap_overflow = 0 # trap overflow or not
• DistributionMapping.strategy = SFC # space filling curve; try KNAPSACK too
• fabarray.mfiter_tile_size = 1024000 8 8 # default for 3D, 1024000 means tiling is off in x-direction; tiling is off by default in 2D;

[nncarlson]

I've given your code changes (parmparse, amrex_init, including cal-back function) a pretty thorough testing now, and it all looks good to me. We really appreciate your willingness to work with us. Thanks!