Add SOP for forking, cloning, genomics pipelines

[mniederhuber]

• write SOP for how to manage and work with genomics pipelines
  • fork repos when doing analysis - personal use
  • branch for specific features and bugs
  • don't branch for analysis
  • don't clone from mckaylab org for analysis

[half-adder]

I think the following process makes sense:

• For each pipeline, users should fork the main repo to their personal account. This allows people to make whatever changes they find relevant for themselves separate from the "gold standard" potentially-public-facing repos.
• Analysis for a given project is done in its own private repository in the mckaylabunc organization
• pipelines can be added as submodules to the analysis directory. This allows the user to have multiple pipelines involved in a single analysis project, while maintaining provenance for each pipeline

Submodules are relatively simple to operate, but using them does require multiple commands so it might be additional mental overhead?

[snystrom]

I am sneaking in to say that submodules are often more trouble than they're worth. In my experience for a published analysis, it doesn't matter whether you can link the pipeline back to a git hash, what you really just want is the source of the pipeline you used embedded into the repo. It makes upgrading the pipeline a little more annoying during an analysis, but it really shouldn't matter that much cause the changes will all get tracked in git & it's trivial to roll them back.

Sincerely,

Someone who suffered using submodules for this purpose early in their PhD. ❤️

[half-adder]

Aha ok that's definitely valuable to hear! I haven't used them very much. So then the above would be the same except cloning repo inside analysis dir instead of using submodules.

[snystrom]

cloning repo inside analysis dir instead of using submodules.

And doing an rm -rf .git inside the repo so you avoid recursive versioning.

[mniederhuber]

Does a project repo have to be in the main mckaylabunc org to use pipelines as submodules?

Personally, I think it will help keep the org clean and easy to navigate if we discourage personal project repos in mckaylabunc, my fear is that we will end up with a lot of "wounded soldiers" so to speak

• with the exception being when a project is at the publication stage. A project repo in a personal account could just be forked to the organization at this point and then made public.
• but maybe we put this to a group vote?

[half-adder]

That sounds reasonable.

Does a project repo have to be in the main mckaylabunc org to use pipelines as submodules?

I don't believe so - it sounds like submodules would be a "if you so dare" situation anyway

[half-adder]

Accidentally closed - sorry