Split workflow into different repos

[Sinclert]

This repository is currently serving 2 different purposes, and may soon serve a different one:

• Purpose 1: Provide a Docker image with all the necessary dependencies to run this Jupyter notebook.
• Purpose 2: Build and publish Docker images (Physics & ML) to run a REANA workflow.
• Purpose 3 (Soon): Provide MLFlow parametrization and tracking features to track how trained models perform (only affecting ML part of the workflow).

Proposal

We could create 3 new repositories:

• madminer-jupyter-env: containing this folder
• madminer-workflow-ml: containing this folder
• madminer-workflow-ph: containing this folder

And leave this repository as a convergence point where madminer-workflow-ml and madminer-workflow-ph are defined as GIT sub-modules (a way of referencing a GIT repository from within a GIT repository).

Example .gitmodules files:

[submodule "madminer-workflow-ml"]
    path = madminer-workflow-ml
    url = git@github.com:scailfin/madminer-workflow-ml.git
    branch = master
[submodule "madminer-workflow-ph"]
    path = madminer-workflow-ph
    url = git@github.com:scailfin/madminer-workflow-ph.git
    branch = master

It would also containing the reana folder with the joint workflow specification.

---

If Scailfin is not the place to create those repositories, then maybe DIANA-HEP (where madminer is hosted), or a brand new organization are better options.

[irinaespejo]

Hi @Sinclert, I think it's a great suggestion. The project is growing and the current repo will be too crowded. I tried using git sub-modules to host a separate repo with the docker images but I didn't get it to work properly. With respect to Scailfin vs DIANA-HEP maybe on the #madminer-on-reana channel they have a better picture on where the repos should go. Thank you for this idea!