search

CliMA / slurmci

CI runner which submits CI tests as Slurm batch jobs
2 stars 2 forks source link

set debug level to 2 for GPU tests #12

Closed vchuravy closed 4 years ago

vchuravy commented 4 years ago

To actually get useful information on kernel failures the only downside should be that the test run a bit more slowly. Alternativly we could grep the stdout of the test and resubmit with -g2 set when we get a kernel failure.

vchuravy commented 4 years ago

Ah the particular failure I was trying to hunt down doesn't occur with -g2, how annoying.

simonbyrne commented 4 years ago

Is this still worth doing?

vchuravy commented 4 years ago

Unsure since it apparently can hide failures at lower debug levels.

simonbyrne commented 4 years ago

It seems that this has a massive performance penalty: running the same test with -g2 seems to give a ~4x slowdown