Auto-generated .yaml

[reinhold-willcox]

Is your feature request related to a problem? Please describe.

Having a yaml file that we update manually with the code defaults reintroduces the same problem we had previously with pythonSubmit. Namely, we need to remind ourselves to update this file, which opens up the possibility that we forget and then the COMPAS defaults are inconsistent across the repo.

This should be a straightforward fix, since the --help output is already nearly in the yaml form. I propose we modify the help output so that it is formatted exactly as the yaml file, so that users can simply run COMPAS --help > config.yaml to get the defaults.

To go a step further, we've had some discussions about the right way to indicate default values while also highlighting other options. I think we should follow the following setup, where all options are shown as commented lines, so users have to uncomment the line and change the value to change the default. Explanations and alternative options (when applicable) are shown as further comments behind where the value is set. I've included below a section of a default matplotlibrc file for matplotlib configs, which shows this implemented.

This is relatively straightforward to implement and would allow us to ensure our defaults are consistently distributed.

 ## ***************************************************************************
 ## * FONT                                                                    *
 ## ***************************************************************************
 ## The font properties used by `text.Text`.
 ## See https://matplotlib.org/api/font_manager_api.html for more information
 ## on font properties.  The 6 font properties used for font matching are
 ## given below with their default values.
 ##
...
 #font.family: Helvetica
 #font.style:   normal
 #font.variant: normal
 #font.weight:  normal
 #font.stretch: normal
 #font.size:     16

 #font.serif:      DejaVu Serif, Bitstream Vera Serif, Computer Modern Roman, New Cen
 #font.sans-serif: DejaVu Sans, Bitstream Vera Sans, Computer Modern Sans Serif, Luci
 #font.cursive:    Apple Chancery, Textile, Zapf Chancery, Sand, Script MT, Felipa, C
 #font.fantasy:    Chicago, Charcoal, Impact, Western, Humor Sans, xkcd, fantasy
 #font.monospace:  DejaVu Sans Mono, Bitstream Vera Sans Mono, Computer Modern Typewr
 ...

 ## ***************************************************************************
 ## * AXES                                                                    *
 ## ***************************************************************************
 ## Following are default face and edge colors, default tick sizes,
 ## default font sizes for tick labels, and so on.  See
 ## https://matplotlib.org/api/axes_api.html#module-matplotlib.axes
 #axes.facecolor:     white   # axes background color
 #axes.edgecolor:     black   # axes edge color
 #axes.linewidth:     0.8     # edge line width
 #axes.grid:          True    # display grid or not
 #axes.grid.axis:     both    # which axis the grid should apply to
 #axes.grid.which:    major   # grid lines at {major, minor, both} ticks
 #axes.titlelocation: center  # alignment of the title: {left, right, center}
 #axes.titlesize:     26      # font size of the axes title
 #axes.titleweight:   normal  # font weight of title
 #axes.titlecolor:    auto    # color of the axes title, auto falls back to
                              # text.color as default value

[jeffriley]

I'm not sure about making the help output the same as the yaml file content - as long as the help is readable. The other possibility is to keep/ensure the help output reader-friendly, and just add another option (e.g. --yaml) that produces a formatted yaml file, or output that can be piped to a file.

My only preference is that the help output be easily readable by a human...

[jeffriley]

This can be done in a matter of a couple of hours if we first generate a template for the yaml file - then adding code to COMPAS is relatively easy. The template would need to be updated whenever new options were added to COMPAS, or existing options removed or renamed, but that's fairly trivial.

[reinhold-willcox]

@jeffriley I spent a few weeks trying unsuccessfully to do this, so I think it's only trivial for you! If you have time to spare to do this, I think it would be useful.

[jeffriley]

I meant updating the template would be trivial - the code changes aren't. See PR #922

Some highlights:

Added two new options: --create-YAML-file and --YAML-template Added two new source files: yaml.cpp and yaml.h

--create-YAML-file, as you might expect, creates a YAML file.
The YAML file created is formatted according to a template.

Users can specify a YAML template using the --YAML-template option. If no template is specified, or the file does not exists or is unreadable, the COMPAS default template will be used - this is declared in yaml.h. Allowing users to specify a template file gives them the flexibility to order and group the options the way they want (most people will accept the default, but some may want to have their own groupings).

The YAML file is formatted as you requested above - mostly :-) I added a "default" field.

The option values written to the YAML file are COMPAS default values, and the line in the YAML file will be commented, UNLESS options were specified on the command line when the YAML file is created, in which case the values will be as specified on the command line, and the line in the YAML file will not be commented. This lets users create project-specific YAML file if the wish (again, most people won't, but some might).

e.g.

./compas --create-yaml-file

will create a YAML file using thedefault template in yaml.h, with all values COMPAS defaults, and all (option definition) lines in the YAML file commented (choices headers are not commented).

./compas --create-yaml-file --logfile-type csv

will create a YAML file using thedefault template in yaml.h, with all values COMPAS defaults, EXCEPT for the value for the --logfile-type option, and all (option definition) lines in the YAML file commented EXCEPT for the definition line for --logfile-type.

I had to modify runSubmit.py to work with a YAML file that had all options in a choice group (i.e. stringChoices, booleanChoices, etc.). I tested the new runSubmit.py against the old YAML file format and it seemed to work ok.

I tried various ways to break the functionality - I'm sure there is a way. It parses a template file, and that template file could have just about anything in it... I've tried to make the code robust - if the template file is poorly formatted and has unexpected characters in it, then you mightn't see the results you expect - but I'm oping it won't actually fail and stop COMPAS. feel free to beat it up as much as you can.

I've added documentation - I added a "PreProcessing" section - and moved "Sampling" under that. The documentation is a bit sparse - feel free to add to it...

[jeffriley]

You need to use the makefile in the PR branch, or modify yours to include the yaml.cpp source file.

On Thu, 9 Mar 2023, 11:14 am Reinhold Willcox, @.***> wrote:

Hey Jeff, I was unable to compile the source for this. I get the following error:

profiling.cpp utils.cpp vector3d.cpp Rand.cpp Options.cpp Log.cpp Errors.cpp BaseStar.cpp Star.cpp MainSequence.cpp MS_lte_07.cpp MS_gt_07.cpp CH.cpp GiantBranch.cpp HG.cpp FGB.cpp CHeB.cpp EAGB.cpp TPAGB.cpp HeMS.cpp HeHG.cpp HeGB.cpp Remnants.cpp WhiteDwarfs.cpp HeWD.cpp COWD.cpp ONeWD.cpp NS.cpp BH.cpp MR.cpp BinaryConstituentStar.cpp BaseBinaryStar.cpp BinaryStar.cpp main.cpp profiling.o utils.o vector3d.o Rand.o Options.o Log.o Errors.o BaseStar.o Star.o MainSequence.o MS_lte_07.o MS_gt_07.o CH.o GiantBranch.o HG.o FGB.o CHeB.o EAGB.o TPAGB.o HeMS.o HeHG.o HeGB.o Remnants.o WhiteDwarfs.o HeWD.o COWD.o ONeWD.o NS.o BH.o MR.o BinaryConstituentStar.o BaseBinaryStar.o BinaryStar.o main.o g++ profiling.o utils.o vector3d.o Rand.o Options.o Log.o Errors.o BaseStar.o Star.o MainSequence.o MS_lte_07.o MS_gt_07.o CH.o GiantBranch.o HG.o FGB.o CHeB.o EAGB.o TPAGB.o HeMS.o HeHG.o HeGB.o Remnants.o WhiteDwarfs.o HeWD.o COWD.o ONeWD.o NS.o BH.o MR.o BinaryConstituentStar.o BaseBinaryStar.o BinaryStar.o main.o -L/lib -L/lib -L/usr/lib/x86_64-linux-gnu/hdf5/serial -Xlinker -rpath -Xlinker /lib -lhdf5_hl_cpp -lhdf5_cpp -lhdf5_hl -lhdf5 -lm -lz -ldl -lsz -lpthread -lgsl -lgslcblas -lboost_filesystem -lboost_program_options -lboost_system -o COMPAS /usr/bin/ld: main.o: in function main': main.cpp:(.text+0x4a7f): undefined reference toyaml::MakeYAMLfile(std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >)' collect2: error: ld returned 1 exit status make: *** [/home/rwillcox/astro//compas/COMPAS//src/Makefile-reinhold:113: COMPAS] Error 1

— Reply to this email directly, view it on GitHub https://github.com/TeamCOMPAS/COMPAS/issues/797#issuecomment-1461066822, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGM5XNQ6CYQX3MNIQTBRRYLW3EOE7ANCNFSM5VRVHLDQ . You are receiving this because you were mentioned.Message ID: @.***>

[reinhold-willcox]

@jeffriley Yup, sorry, I realized right after I commented.