search

facebookresearch / dora

Dora is an experiment management framework. It expresses grid searches as pure python files as part of your repo. It identifies experiments with a unique hash signature. Scale up to hundreds of experiments without losing your sanity.
MIT License
262 stars 24 forks source link

[Feature request] Export grid tree table to LaTeX/csv #61

Open Mattias421 opened 7 months ago

Mattias421 commented 7 months ago

❓ Questions

Hi all,

I think the Tree Table feature is great for monitoring, but also presenting final results, unfortunately, there doesn't seem to be an easy way to export to LaTeX or flatten the tree and save as csv.

Is this feature already secretly implemented or should I get coding :)

Best, Mattias

adefossez commented 6 months ago

Not a feature currently but I would be mostly against integrating that upstream, everyone will want different things and I don't want to have to support each use case but keep Dora minimalistic. I think I put some doc in the readme on how to list experiments in a grid and recover metrics and config. For instance xp.delta can be a good thing to look at to map short configs differences and then xp.link.load() + reading xp.link.history.

adefossez commented 6 months ago

Also the how information is stored on disk is pretty straightforward. In the dora output dir, go to the grids subfolder, you can then just list your grid folder there, it will contain one symlink per experiment in the grid whose name is the XP signature. For instance:

from yourproject import train
grid = train.main.dora.dir / 'grids' / grid_name
for child in grid.iterdir():
    xp = train.main.get_xp_from_sig(child.name)
    xp.link.load()
    # look at xp.delta and xp.link.history
Mattias421 commented 5 months ago

Looping through the grid this way is an excellent way to process results, and pandas support good conversion from structured data to presentable latex tables, cheers!