search

firedrakeproject / firedrake

Firedrake is an automated system for the portable solution of partial differential equations using the finite element method (FEM)
https://firedrakeproject.org
Other
482 stars 157 forks source link

BUG: Performance regression on CI #3603

Open JDBetteridge opened 4 weeks ago

JDBetteridge commented 4 weeks ago

Describe the bug We have hit yet another performance regression! This PR showed a performance regression:

PR: https://github.com/firedrakeproject/firedrake/pull/3577

CI (tests 1hr): https://github.com/firedrakeproject/firedrake/actions/runs/9288922418/job/25561549540

When merged to master the same behaviour was observed:

CI (tests 52mins): https://github.com/firedrakeproject/firedrake/actions/runs/9316886159/job/25646023795

When that commit is reverted the execution time drops again:

CI (tests 42mins): https://github.com/firedrakeproject/firedrake/actions/runs/9319667894/job/25654727405?pr=3602 (42mins is longer than the scheduled tests which run for 35mins, but usually these have no resource contention https://github.com/firedrakeproject/firedrake/actions/runs/9239093106/job/25417813619)

This PR is keeping track of various fixes https://github.com/firedrakeproject/firedrake/pull/3602

I'm going to wait for the next scheduled CI run to see how significant this issue is.

Steps to Reproduce Steps to reproduce the behavior:

  1. Use CI

Expected behavior Same performance as before.

Error message N/A

Environment:

Additional Info Add any other context about the problem here.

JDBetteridge commented 3 weeks ago

Maybe everything is fine... https://github.com/firedrakeproject/firedrake/actions/runs/9334091136/job/25727536478

JDBetteridge commented 3 weeks ago

These regressions may be related to the disks on the runners being almost full!

I will continue to monitor the situation