valassi
commented
3 years ago Yesterday I created a first WIP PR #245 for the "step 1". One important (unexpected) issue is that the current code generation in Madgraph seems to be not reproducible, which in my opinion is a major problem that prevents easy validation of any new developments (both those we already have, such as vectorized c++ or kokkos, and any new development to come). Maybe this is not complex to fix (some python dictionaries to replace by sorted ones?), maybe it is. I will investigate when I have time.
I put down a few ideas following this afternoon's code generation workshop https://indico.cern.ch/event/1061524/ about how I suggested I would structure the repo (and also specific steps I plan to implement for cuda/c++)
What I would create is a new epochX directory, next to epoch1 and epoch2.
The plan is to backport my vectorization changes (now the latest are in epoch1/cuda/ee_mumu) to epochX/cuda.
This epochX would be an evolving directory (i.e. not a semi-fixed one like epoch1/2), containing both manually developed code and code-generating code infrastructure.
In a "steady state", when all new manual developments have been backported to the code generating code, one should be able to run the code generating code and obtain EXACTLY the same code as that which has been manually developed and committed to the repo. This requires a functioning code generating infrastructure (eg with only a few plugin files in github, but relying on an "external" installation of Mg5aMC somewhere configurable), and some relocation and diff scripts: essentially yo generate new code in a tmp area, then you compare it to what is committed in the repo. It may also require some beautifier to bypass trivial formatting differences (to be seen).
Beyond the steady state, one develops manual code and commits that to the github repo. At this stage, you have in github a code generating plugin that is "older" than the manual developments, and that is perfectly ok. At some point, you do the effort of backporting the new manual changes into the code generating plugin, and ensure that you reach the steday state again. Then you can iterate and do new manual developments.
Specifically for the "cuda" directory (native cuda, plus vectorized c++, with ifdefs for the moment), I would imagine working on the following steps in sequence:
Create the basic infrastructure, with "manual" code and code generating code. Ensure that you get the external MG5aMC and diff scripts working. In practice, start with Olivier's current bazaar code generatoing code, add that as a plugin to github. Generate processes (eemumu and ggttgg? probably both) with that code generating code, commit that to github as the first "manual" version. This would be clearly different from the latest epoch1/eemumu (missing all vectorization), but also from epoch2/ggttgg (I checked today, there are small differences). This is a steady state. Make a PR for this.
Replace epochX/ggttgg by the manual code that is currently in epoch2/ggttgg. This is beyond steady state, Start doing actual work on the code generating plugin, ensure that eventually the plugin is able to generate exactly the same ggttgg code. Make sure that the new plugin also works fine on eemumu: generate a new eemumu version and just check it looks ok, commit that to github. This is again a steady state, for both ggttgg and eemumu. Make a second PR.
Attack vectorization. Replace epochX/eemumu by the manual code that is currently in epoch1/eemumu. This is far beyond steday state. Do the (potentially complex) part of backporting vectorization to code generating code. Here you might also need the beautifier, if not used already before. Ensure that you are able to get exactly the same eemumu code from code generation as you have in the manual version. If done, you are in steady state for eemumu. Make a third PR.
Here it gets interesting. Apply the latest epochX code generator to ggttgg. This should produce vectorized code for ggttgg, but it is likely to be missing some bits. Iterate on both sides, manual ggttgg code and code generating code, until you get something that actually works well for ggttgg, and can be reporoduced exactly from code generation. If done, you are in the steady state for ggttgg. Make a fourth PR.
At this point you probably have a more recent code generator than you had in step 3, and you should make sure that eemumu also looks ok. Iterate a bit on all sides (manual eemumu, manual ggttgg, and code generating code), until both processes look reasonable and can be fully auto generated. You are now in the final steady state for the backport of previous cuda/c++ code. Make a fifth PR.
From here onwards, go on with the normal development model outlined at the very beginning: more changes in manual cuda/c++ for eemumu and/or ggttgg (probably ggttgg will be the main focus by now, and we can fully forget eemumu?). At some point (either here, or maybe already in step 4) you probably need to do something special for color algebras. Anyway, from here onwards there are many more developments, eg Fortran integration, comparison to other backends, etc.
Concerning code generating code for cuda vs kokkos and others: I would probably do all of the steps 1-5 in isolation. Maybe the only thing that it makes sense to have in common at the beginning in step 1 is the external MG5aMC installation, and possibly the tooling to do code generation tests and diffs (and possibly beautification). But the plugins themselves in the code generating code I would initially keep separate for cuda/c++ and kokkos (and alpaka, sycl etc). It looks already quite complex like this... Anyway, this can be rediscussed.
Voila, a long brain dump to clear my own ideas/plans, and explain to others what I'd suggest... feedback welcome of course! Andrea