mlcommons / modelbench

Run safety benchmarks against AI models and view detailed reports showing how well they performed.
https://mlcommons.org/ai-safety/
Apache License 2.0
50 stars 8 forks source link

Always print progress bar #420

Closed bkorycki closed 2 weeks ago

bkorycki commented 2 weeks ago

Output with --machine-reports --parallel true flags from before: Screenshot 2024-08-27 at 12 23 27 PM

After this change: Screenshot 2024-08-27 at 12 24 20 PM

Let me know if this is what we want.

github-actions[bot] commented 2 weeks ago

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

wpietri commented 2 weeks ago

Not sure how hard this would be, but what I was hoping for with --machine-reports was no progress bar, the JSONL progress update, and either regular logging or structlog-style JSONL logging.

bkorycki commented 2 weeks ago

Okay I made quite a few changes @wpietri @dhosterman

dhosterman commented 2 weeks ago

I just tested this against the latest changes in modellab and it works great! It's still showing progress bars for the dependency downloads, but that'll require downstream work in modelgauge to fix and doesn't impede using the progress data provided.