Daily Scheduled Test Failure

mlcommons / modelgauge

Make it easy to automatically and uniformly measure the behavior of many AI Systems.

https://mlcommons.org/ai-safety/

Apache License 2.0

26 stars 7 forks source link

Daily Scheduled Test Failure #520

Closed github-actions[bot] closed 3 months ago

github-actions[bot] commented 3 months ago

❌ Daily Scheduled Test Failure ❌

Commit: 95798681356b8741fa28ca9d27b3b1d92c27bc34 Run Id: 10434333654

wpietri commented 3 months ago

Also #521

wpietri commented 3 months ago

And #522

rogthefrog commented 3 months ago

(at least) two failure modes:

third-party timeout (we want to know about those)
no space left on device (we want to fix those)

Looking at dependency managers to see if the venv size can be shrunk (fawltydeps, deptry, and creosote).

rogthefrog commented 3 months ago

Deptry and fawltydeps don't report any unused imports of significant importance or size.

The venv built from scratch locally is 374MB.

Going back to look at ways to ensmallen the runner.

rogthefrog commented 3 months ago

@wpietri there goes nothing

wpietri commented 3 months ago

I'm optimistic!

On Mon, Aug 26, 2024 at 7:11 PM Roger @.***> wrote:

@wpietri https://github.com/wpietri there goes nothing

— Reply to this email directly, view it on GitHub https://github.com/mlcommons/modelgauge/issues/520#issuecomment-2311324951, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAUMW7CP3ANS4VQ7WMNTY3ZTO737AVCNFSM6AAAAABMVT5LO2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMJRGMZDIOJVGE . You are receiving this because you were mentioned.Message ID: @.***>