madgraph5 / madgraph4gpu

GPU development for the Madgraph5_aMC@NLO event generator software package
30 stars 33 forks source link

Initial skeleton of a more comprehensive CI test suite #792

Closed valassi closed 10 months ago

valassi commented 10 months ago

Hi @roiser @hageboeck @oliviermattelaer @zeniheisser as discussed this is a PR with an initial skeleton of a more comprehensive CI test suite.

Presently this does

Amongst things to add

Anyway: for the moment these tests wich are in the PR should all succeed. So this should already be a first good approximation of something to check that we are not introducing new issues with some CODEGEN changes.

(Ah, the codegen test presently compares the code to the git repo... but I will disable this for the moment. I can take care pf popoulating the generated code in the repo if you want. But it would be great if you can run at least these tests when making new changes to codegen).

I am back in a few days and we can discuss this later on. Thanks! Andrea

valassi commented 10 months ago

Here you see that the full test suite is successful https://github.com/madgraph5/madgraph4gpu/actions/runs/6757359626 The full initial build and test of pp_tt012j.mad takes 19 minutes when not cached https://github.com/madgraph5/madgraph4gpu/actions/runs/6757359626/job/18367850043

I have now done a forced push just to retrigger the workflow (after it is merged, it will be possible to MANUALLY trigger it from https://github.com/madgraph5/madgraph4gpu/actions/workflows/testsuite_allprocesses.yml). The full build and test of pp_tt012j.mad when it is cached via ccache is now just over 2 minutes! https://github.com/madgraph5/madgraph4gpu/actions/runs/6757482392/job/18368091997

In other words, when the CODEGEN changes are minor and do not result in real code changes, these tests are fast. However when the generated code does change, one must rebuild it from scratch an dit takes longer.

oliviermattelaer commented 10 months ago

Thanks for this Andrea having tests is always a good things. So I approve. Now we will learn in the long run which tests are "problematic" to maintain and which tests we are missing. Both will need to be fixed in due time (if needed obviously).

Thanks for this,

Olivier

valassi commented 10 months ago

Thanks Olivier very good I merge this then :-)

I have a few other things in the pipeline, this helps me get things out of the way.

NB THE CI TESTS WILL BE A BIT LONGER NOW (20 minutes? I can disable the longest ones if this is a nuisance, initially)