ENH: Simplify config in logs

[larsoner]

If we follow this premise, we could create a single Excel sheet where one row corresponds to one line or one setting in the config file that was used

_Originally posted by @hoechenberger in https://github.com/mne-tools/mne-bids-pipeline/pull/865#discussion_r1508941664_

FWIW I did start looking into adding just a config sheet with something like:

@@ -317,10 +292,18 @@ def save_logs(*, config: SimpleNamespace, logs: list[pd.Series]) -> None:
             mode="a" if append else "w",
             if_sheet_exists="replace" if append else None,
         )
+        assert isinstance(config, SimpleNamespace), type(config)
+        cf_df = dict()
+        for key, val in config.__dict__.items():
+            cf_df[key] = json_tricks.dumps(val, sort_keys=False)
+        cf_df = pd.DataFrame(cf_df)
         with writer:
+            # Config first then the data
+            cf_df.to_excel(writer, sheet_name="config", index=False)
             df.to_excel(writer, sheet_name=sheet_name, index=False)

but it fails because the config at this level isn't trimmed to the only relevant entries. But this should be fixable in principle if we want to go this route.

[hoechenberger]

but it fails because the config at this level isn't trimmed to the only relevant entries.

Do you mean, it contains all values and not just those that were adjusted by the user?

[larsoner]

Do you mean, it contains all values and not just those that were adjusted by the user?

No I mean it contains everything in the user's config.py namespace, not just the valid config options. So like for ERP_CORE it contains argparse from the import argparse line in that test config. I'll try trimming just to the valid config options, it should fix it...