Config implementation

[stefsmeets]

This PR updates the code to use the new style config. To help with the transition, I added a compatibility layer with the old style config, so that a dict lookup (like OPTIONS['...']) will still work for the most part, but the idea is to completely move over to the new format.

I added a superannoying warning to help with motivation :smiling_imp:

I also updated all the old input tomls to the new format.

The most tricky bits are around the years, which changed from yr -> Gyr, and modules, which changed from integers to enums. Please pay extra attention to those.

Closes #74

TODO

• [x] Get dummy test to pass
• [x] Get other tests to pass
• [x] Add some tests for config reader
• [x] star age changed (yr -> Gyr), baraffe and spada expect different conversions
• [x] Test non-dummy modules (zephyrus, spider, mors, janus, ...)

[stefsmeets]

Got the loop working

[stefsmeets]

Code works, but I get hit with ERROR tests/integration/test_dummy_integration.py::test_dummy_run - Exception: Could not find solution for volatile abundances (max attempts reached)

Any ideas where to look @nichollsh @timlichtenberg?

Click to see log

```console pytest tests/integration/test_dummy_integration.py::test_dummy_run ``` ``` INFO fwl.proteus.interior.dummy:dummy.py:20 Running dummy interior... DEBUG fwl.proteus.interior.timestep:timestep.py:38 Previous step size: 2.11e+03 yr INFO fwl.proteus.interior.timestep:timestep.py:65 Time-stepping intent: speed up INFO fwl.proteus.interior.timestep:timestep.py:90 New time-step is 2.32e+03 years DEBUG fwl.proteus.interior.wrapper:wrapper.py:54 T_magma = 2247.1 K DEBUG fwl.proteus.interior.wrapper:wrapper.py:55 Phi_global = 0.547 INFO fwl.proteus.utils.helper:helper.py:25 --------------------------------------------------- INFO fwl.proteus.proteus:proteus.py:324 Stellar flux management... INFO fwl.proteus.proteus:proteus.py:339 Updating instellation and radius DEBUG fwl.proteus.proteus:proteus.py:380 Instellation change: +4.8442e-04 W m-2 (to 4dp) INFO fwl.proteus.proteus:proteus.py:421 New spectrum not required at this time INFO fwl.proteus.utils.helper:helper.py:25 --------------------------------------------------- INFO fwl.proteus.utils.escape:escape.py:37 Running dummy escape... DEBUG fwl.proteus.utils.escape:escape.py:60 H mass ratio = 3.48e-01 DEBUG fwl.proteus.utils.escape:escape.py:60 C mass ratio = 3.48e-01 DEBUG fwl.proteus.utils.escape:escape.py:60 N mass ratio = 3.01e-03 DEBUG fwl.proteus.utils.escape:escape.py:60 S mass ratio = 3.01e-01 INFO fwl.proteus.proteus:proteus.py:450 Bulk escape rate: 6.31e+11 kg yr-1 = 2.00e+04 kg s-1 INFO fwl.proteus.utils.helper:helper.py:25 --------------------------------------------------- INFO fwl.calliope.solve:solve.py:379 Solving for equilibrium partial pressures at surface DEBUG fwl.calliope.solve:solve.py:380 target masses: {'H': 9.299946099657987e+20, 'C': 9.299935302042133e+20, 'N': 8.062153522871222e+18, 'S': 8.062152745248075e+20} DEBUG fwl.calliope.solve:solve.py:417 Initial guess attempt number = 761 DEBUG fwl.calliope.solve:solve.py:435 Residuals: [np.float64(-4.351047779793919e-09), np.float64(-0.048460087244712825), np.float64(-3.6224354758238815e-05), np.float64(-8.120052889438231e-08)] INFO fwl.calliope.solve:solve.py:460 H2O : 51.71 bar (3.54e-03 VMR) INFO fwl.calliope.solve:solve.py:460 CO2 : 7530.44 bar (5.15e-01 VMR) INFO fwl.calliope.solve:solve.py:460 H2 : 8.98 bar (6.15e-04 VMR) INFO fwl.calliope.solve:solve.py:460 CH4 : 0.00 bar (1.03e-07 VMR) INFO fwl.calliope.solve:solve.py:460 CO : 6935.19 bar (4.75e-01 VMR) INFO fwl.calliope.solve:solve.py:460 N2 : 87.60 bar (5.99e-03 VMR) INFO fwl.calliope.solve:solve.py:460 S2 : 0.01 bar (9.68e-07 VMR) INFO fwl.calliope.solve:solve.py:460 SO2 : 0.21 bar (1.43e-05 VMR) INFO fwl.proteus.utils.helper:helper.py:25 --------------------------------------------------- INFO fwl.proteus.atmos_clim.dummy:dummy.py:19 Running dummy atmosphere... WARNING fwl.proteus.atmos_clim.dummy:dummy.py:38 Rayleigh scattering is enabled but it will be neglected INFO fwl.proteus.atmos_clim.dummy:dummy.py:40 Gamma = 0.7000 INFO fwl.proteus.atmos_clim.dummy:dummy.py:67 Calculating fluxes with dummy atmosphere and CBL INFO fwl.proteus.atmos_clim.dummy:dummy.py:83 Found solution after 4 iterations INFO fwl.proteus.atmos_clim.dummy:dummy.py:103 Resultant values: INFO fwl.proteus.atmos_clim.dummy:dummy.py:104 T_surf = 2.195e+03 K INFO fwl.proteus.atmos_clim.dummy:dummy.py:105 F_atm = 1.047e+04 W m-2 INFO fwl.proteus.atmos_clim.dummy:dummy.py:106 F_olr = 1.066e+04 W m-2 INFO fwl.proteus.atmos_clim.dummy:dummy.py:107 F_sct = 2.317e+01 W m-2 INFO fwl.proteus.utils.helper:helper.py:25 --------------------------------------------------- DEBUG fwl.proteus.utils.coupler:coupler.py:193 Extending helpfile with new row DEBUG fwl.proteus.utils.coupler:coupler.py:211 Writing helpfile to CSV file INFO fwl.proteus.utils.coupler:coupler.py:73 Runtime info... INFO fwl.proteus.utils.coupler:coupler.py:74 System time : 2024-10-15_14-14-05 INFO fwl.proteus.utils.coupler:coupler.py:75 Model time : 2.44e+04 yr INFO fwl.proteus.utils.coupler:coupler.py:76 T_surf : 2194.774 K INFO fwl.proteus.utils.coupler:coupler.py:77 T_magma : 2247.136 K INFO fwl.proteus.utils.coupler:coupler.py:78 P_surf : 1.46e+04 bar INFO fwl.proteus.utils.coupler:coupler.py:79 Phi_global : 5.47e-01 INFO fwl.proteus.utils.coupler:coupler.py:80 Instellation : 1.03e+03 W m-2 INFO fwl.proteus.utils.coupler:coupler.py:81 F_int : 1.16e+04 W m-2 INFO fwl.proteus.utils.coupler:coupler.py:82 F_atm : 1.05e+04 W m-2 INFO fwl.proteus.utils.coupler:coupler.py:83 |F_net| : 1.08e+03 W m-2 INFO fwl.proteus.proteus:proteus.py:550 Check convergence criteria INFO fwl.proteus.proteus:proteus.py:284 INFO fwl.proteus.utils.helper:helper.py:21 =================================================== INFO fwl.proteus.proteus:proteus.py:286 Loop counters INFO fwl.proteus.proteus:proteus.py:287 init total steady INFO fwl.proteus.proteus:proteus.py:288 2/2 0038/9000 -1/3 INFO fwl.proteus.utils.helper:helper.py:25 --------------------------------------------------- INFO fwl.proteus.interior.dummy:dummy.py:20 Running dummy interior... DEBUG fwl.proteus.interior.timestep:timestep.py:38 Previous step size: 2.32e+03 yr INFO fwl.proteus.interior.timestep:timestep.py:65 Time-stepping intent: speed up INFO fwl.proteus.interior.timestep:timestep.py:90 New time-step is 2.55e+03 years DEBUG fwl.proteus.interior.wrapper:wrapper.py:54 T_magma = 2188.2 K DEBUG fwl.proteus.interior.wrapper:wrapper.py:55 Phi_global = 0.488 INFO fwl.proteus.utils.helper:helper.py:25 --------------------------------------------------- INFO fwl.proteus.proteus:proteus.py:324 Stellar flux management... INFO fwl.proteus.proteus:proteus.py:339 Updating instellation and radius DEBUG fwl.proteus.proteus:proteus.py:380 Instellation change: +5.3286e-04 W m-2 (to 4dp) INFO fwl.proteus.proteus:proteus.py:421 New spectrum not required at this time INFO fwl.proteus.utils.helper:helper.py:25 --------------------------------------------------- INFO fwl.proteus.utils.escape:escape.py:37 Running dummy escape... DEBUG fwl.proteus.utils.escape:escape.py:60 H mass ratio = 3.54e-01 DEBUG fwl.proteus.utils.escape:escape.py:60 C mass ratio = 3.37e-01 DEBUG fwl.proteus.utils.escape:escape.py:60 N mass ratio = 3.07e-03 DEBUG fwl.proteus.utils.escape:escape.py:60 S mass ratio = 3.07e-01 INFO fwl.proteus.proteus:proteus.py:450 Bulk escape rate: 6.31e+11 kg yr-1 = 2.00e+04 kg s-1 INFO fwl.proteus.utils.helper:helper.py:25 --------------------------------------------------- INFO fwl.calliope.solve:solve.py:379 Solving for equilibrium partial pressures at surface DEBUG fwl.calliope.solve:solve.py:380 target masses: {'H': 9.299940355680237e+20, 'C': 8.849254198820772e+20, 'N': 8.061856532346801e+18, 'S': 8.062147146203687e+20} ```

[nichollsh]

This usually comes about if the outgassing model (CALLIOPE) is given unreasonable conditions, such as very low temperatures or zero-valued mass inventories. I can see that the T_magma and target masses in the log look reasonable, and this configuration should work fine since we used it in the previous tests.

I would suggest checking that the parameters which are passed to the outgassing code are reasonable. For example, maybe it's being provided with M_mantle=0, or the core fraction is unset, or something along those lines.

[stefsmeets]

Sounds like it could be this bit where I made quite some changes. Do you see anything odd?

https://github.com/FormingWorlds/PROTEUS/blob/95ae68144a6e36bb1202de1cdebbdada544b4502/src/proteus/proteus.py#L472-L501

EDIT: Looks like there is a typo in the radius EDIT2: That fixed it, thanks for saving me many hours of debugging @nichollsh :1st_place_medal:

[nichollsh]

No problem! Glad that this works.

[stefsmeets]

Alright, next question for @nichollsh @timlichtenberg, any thoughts about which parameter I should look at? I'm very close to get this to work, but I think I messed up some parameter that affects the surface temp (starts at 3500 instead of 3250ish) which means the transition starts slightly earlier in the current tests.

Actual:

Expected:

Diff:

[nichollsh]

Could be something to do with the init loops. The way the initial temperature is set differs between the dummy interior and SPIDER (we should think about this more deeply elsewhere - ideally we'd just set T_surf in the config file and that would be accepted by all the interior modules).

The dummy model should enforce an initial temperature by setting T_magma equal to 3500 during every step of the init phase. This is hardcoded inside the RunDummyInt function, and applies when IC_INTERIOR==1, which is true when loop_counter["init"] < loop_counter["init_loops"].

If you've changed the looping stuff, maybe that could explain this?

[stefsmeets]

Thanks, the issue was that the surface state was set to 'skin' instead of 'fixed' in the config 👍

[stefsmeets]

Dummy tests are passing now 🚀

[stefsmeets]

@lsoucasse @nichollsh The tests are passing and I think I covered many modules already. Could you help and have a test with some of your favorite modules (spider, zephyrus, janus, etc) to find the last places where the config should be updated?

[nichollsh]

Sure! I'll run some models by writing some configs "by hand" to see if we get the expected output.

[nichollsh]

@stefsmeets, when I try to run proteus with the default config it raises an error:

Traceback (most recent call last):
  File "/home/n/nichollsh/nobackups/miniforge3/envs/proteus/bin/proteus", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/home/n/nichollsh/nobackups/miniforge3/envs/proteus/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/n/nichollsh/nobackups/miniforge3/envs/proteus/lib/python3.12/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/n/nichollsh/nobackups/miniforge3/envs/proteus/lib/python3.12/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/n/nichollsh/nobackups/miniforge3/envs/proteus/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/n/nichollsh/nobackups/miniforge3/envs/proteus/lib/python3.12/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/n/nichollsh/PROTEUS/src/proteus/cli.py", line 75, in start
    runner.start(resume=resume)
  File "/home/n/nichollsh/PROTEUS/src/proteus/proteus.py", line 92, in start
    ValidateInitFile(self.directories, self.config)
  File "/home/n/nichollsh/PROTEUS/src/proteus/utils/coupler.py", line 250, in ValidateInitFile
    if (config[key_pp] > 0.0) and (config["solvevol_use_params"] > 0):
                                   ~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/n/nichollsh/PROTEUS/src/proteus/config/_config.py", line 40, in __getitem__
    val = conv(self)
          ^^^^^^^^^^
  File "/home/n/nichollsh/PROTEUS/src/proteus/config/_compatibility.py", line 41, in _solvevol_use_params
    raise ValueError("Equivalent to `delivery.initial = 'elements'`")
ValueError: Equivalent to `delivery.initial = 'elements'`

[nichollsh]

Do we still need this function ValidateInitFile()? Presumably these checks should be made inside the _config/ Python files.

[stefsmeets]

I will pick up validation after, please add any validation related tasks to #200

solvevol_use_params should be factored out, since it is no longer needed based on your comment in https://github.com/FormingWorlds/PROTEUS/issues/74#issuecomment-2385375968 , can this bit be deleted?

[nichollsh]

I would say we can delete it, yeah. If you push to this branch I will pull it again and try to run some more tests.

[lsoucasse]

Hi @stefsmeets, I don't have the log print on the terminal anymore with this branch. How can I retrieve it? The log file in the output folder is also empty.

[stefsmeets]

@lsoucasse Which workflow are you running? I turned it off for the dummy toml, because it gives too much spam with pytest.

https://github.com/FormingWorlds/PROTEUS/blob/75392e2cfd45a76ae68c01d933cb77d9461704a8/input/dummy.toml#L29

[nichollsh]

@stefsmeets another minor issue, now that the solvevol one is fixed:

  File "/home/n/nichollsh/PROTEUS/src/proteus/atmos_clim/janus.py", line 54, in InitAtm
    raise Exception("Invalid tropopause option '%d'" % config["tropopause"])

The match statement in this function needs to be changed to match strings, rather than integers for the tropopause type.

[lsoucasse]

For the escape/zephyrus we miss two keys:

• efficiency_factor: float
• escape_el_tidal_correction: bool

I guess the idea is to add them to the class and the compatibility map? Is that all?

[nichollsh]

The default case seems to run fine now for me. However, the HD63433d case with AGNI crashes.

  File "/home/n/nichollsh/PROTEUS/src/proteus/atmos_clim/agni.py", line 160, in init_agni_atmos
    elif chem_type >= 2:
         ^^^^^^^^^^^^^^
TypeError: '>=' not supported between instances of 'NoneType' and 'int'

[stefsmeets]

@lsoucasse It's not immediately clear to me where these come from, because these are not in #74 . Can we add these in a follow-up PR? Together with the omega/pctle if needed?

[lsoucasse]

@lsoucasse It's not immediately clear to me where these come from, because these are not in #74 . Can we add these in a follow-up PR? Together with the omega/pctle if needed?

@stefsmeets , these parameters are specific to escape.toml. But I agree they can be addressed is a separate PR. We'll need to go through all option key anyway to resolve the warnings.

[stefsmeets]

I got default.toml running without any errors, and tests are also passing. I didn't have a chance to test agni yet.

I sorted out many (all?) of the int -> str conversions.

Follow-up:

• [ ] Escape config (omega, rot_pctle)
• [ ] Sort out remaining config related deprecation warnings

[nichollsh]

Question: are we planning on eventually phasing out the need for this conversion layer? Are we planning to access the variables like config.atmos_clim.tropopause rather than as config["tropopause"]? Obviously it's useful in the meantime while we adapt the code to the new config format.

Edit: Ignore this! I can see now the very annoying warning that you describe, which answers this question.

[lsoucasse]

~Question: are we planning on eventually phasing out the need for this conversion layer? Are we planning to access the variables like config.atmos_clim.tropopause rather than as config["tropopause"]? Obviously it's useful in the meantime while we adapt the code to the new config format.~

Edit: Ignore this! I can see now the very annoying warning that you describe, which answers this question.

I think it is still worth coordinating to address these warnings, maybe module by module.

[stefsmeets]

Thanks for the feedback!

@nichollsh I'm having some issues getting agni to work (need to reinstall agni), but if it works for you and you are happy with the changes, then this PR can be merged.

[nichollsh]

having some issues getting agni to work

Maybe we can discuss this at the next proteus meeting.