search

SBNSoftware / sbnci

Packages, modules and scripts for the SBN continuous integration system
2 stars 4 forks source link

Fix/Verify any warnings and errors in current SBND CI tests #13

Open wesketchum opened 3 years ago

wesketchum commented 3 years ago

Check thresholds/etc.

Should be done ahead of 13 Aug release.

This may be more appropriate to an SBND issues/PR: link here and close this as needed.

wesketchum commented 3 years ago

Assigning @chilge and @henrylay97 : feel free to just say 'done' and close or comment on anything you know that's outstanding.

chilge commented 3 years ago

it looks like the last automatic daily build (a bit more than an hour ago) failed at the unit test stage. see the dashboard here. both geometry_sbnd and geometry_sbnd_test fail. the specific error message is

%MSG-e WireIDIntersectionCheck: GeometryTest:geotest@BeginJob 10-Aug-2021 17:16:31 CDT ModuleBeginJob Comparing two wires on different TPCs: return failure.

The previous set of automatic CI builds passed only for e20:prof. The warnings for e20:debug, c7:prof, c7:debug are mostly memory/cpu usage out of range and differences in product sizes.

henrylay97 commented 3 years ago

So with the rate of development means that there are almost constantly some levels of warnings on the dashboard. This is all closely monitored by @pgreen135 and myself. The test we try to keep green at all times is e20:prof. Any product differences in these tests are genuine with respect to the references (which are generated using e20). Currently develop is clean on e20:prof.

The c7 builds are more used to check that the code definitely compiles in clang as well. This occasionally throws up errors in the build stage we don't see for e20 allowing us to pick up on unused variables for example. Warnings from the ci_tests box for c7 are to be expected and aren't a worry. Failures would be.

The failure that @chilge mentions above is actually from a build for PR #141 in sbndcode. It sounds like the reasons are understood so I'll look at triggering without these tests in the suite.

wesketchum commented 3 years ago

Thanks guys. I'll leave this open, but am going to move it to a new "2021B_XX" project so we can aim to close it out once all references have been updated with the new release. Let me know if you disagree at all.

wesketchum commented 3 years ago

Bumping here. I think we have the (hopefully final) patch release, so please remember to update the references when complete (if not already done).

henrylay97 commented 3 years ago

Apologies I was away. The CI tests against develop so the references are based on that. I am currently updating the references to match some changes from v09_29_00. As always we only ever expect to see the e20:prof builds come back completely clean (all green ticks), some levels of warnings for other compilers (usually on memory/time allocation) are to be expected.