stress: slow pkgs don't get stressed for even 1 iter

[tbg]

See https://github.com/cockroachdb/cockroach/issues/101894#issuecomment-1624346881

For packages whose tests are slow / fall over under stress, we sometimes spend an entire hour but don't get through a single iteration of the pkg. In such cases, the test runner aborts. So we don't get the full benefit of stressing.

We should at least surface such cases. There are two obvious changes we can consider: fail such tests and the owning team(s) need to break them down so that they can get through one iteration. Or, run at least one interation anyway, even if it takes, say, 8h. If a pkg exceeds that, though, we are likely going to have to fall back to the first suggestion, so maybe that is the better one in the first place.

Jira issue: CRDB-29519

[blathers-crl[bot]]

Hi @tbg, please add a C-ategory label to your issue. Check out the label system docs.

While you're here, please consider adding an A- label to help keep our repository tidy.

_{:owl: Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.}

[blathers-crl[bot]]

cc @cockroachdb/test-eng