relation between dagobah and the admin section of this repo

[bgruening]

I would like to discuss the relation between the https://github.com/gvlproject/dagobah-training Admin training initiative and this repository and the included admin training.

xref: https://github.com/galaxyproject/training-material/issues/123 ping @martenson @natefoo @Slugger70

[Slugger70]

Yes, I would too.. I envisaged something like this:

[bgruening]

[martenson]

I have reworked the dagobah-training repo recently to follow the branch-per-training paradigm. It is now owned by galaxyproject, branches automatically built by jenkins, and we host all training materials simultaneously on github pages.

I outlined the approach to learn | teach using that repository in the new readme

What do you think?

Sorry I did not see this issue sooner.

[martenson]

I think the main issues with the approach on the schema above from @Slugger70 are:

The contribute back part - In my experience people tend to not care that much after delivering the training (hell, even our own 2017 Melbourne did not get propagated back). I think it would be better to leave this part for the future people preparing the training. The previous materials will always be available to them and they are invested in improving/consolidating them into the up to date version.

No control over forks (materials can disappear at any time).

cc @jmchilton

[bgruening]

I also favor the branch-per-training paradigm, but I'm wondering here if this repo should host training material at all and if so, to which degree. Why not having everything in one repo, etc ... what you are missing from this repo, which infrastructure we want to share ...

[martenson]

@bgruening atm I do not think we are missing any infrastructure provided by the training-material repo. The trainings have very different audiences and I do not think multiple approaches are bad per se. To be frank I do not see enough benefits in having it under a shared roof.

[hrhotz]

Each galaxy training has a different audience: people want to learn Bioinformatics, or want to learn about galaxy itself, or want to learn how to write tools, or want to learn how to be a galaxy admin. So I am in favor of a 'shared roof'. Though, I don't mind having two different repositories. However, in this case, I strongly suggest to remove all the admin stuff (i.e. training-material/Admin-Corner/) from the training-material repo. And to make the separation clear: move training-material/Dev-Corner/ as well. To make sure, there is no duplication in efforts.

....writing these lines, just makes me think, that it really should all go into one repository, IMHO.

[nsoranzo]

@martenson IMO, the goals of the training-material repository are:

• to share high quality training material about Galaxy, not restricted to user-dedicated topics
• to have a central place where trainers can work together on this material
• to support the Galaxy Training Network and the creation of a trainers community
• to have a shared infrastructure (common templates, build system) to transform the sources into various formats (PDF, presentations, handouts,...) using a coherent (but adjustable) layout.

I think that the fact the Galaxy Team uses a separate repo for its training materials undermines many of these goals. Sorry if that sounds blunt, but "not seeing enough benefits" doesn't seem a good reason to keep stuff separated and duplicated. If there are issues that block you from contributing to this repo, please tell us.

[jmchilton]

I think that the fact the Galaxy Team uses a separate repo for its training materials undermines many of these goals.

I'd caution against using the term "Galaxy Team" here - I can count at least 5 people funded by "the NIH grant" that have contributed to the training-material repository and not so with the dagobah material - also that material originated with the GVL I believe. You are the big kid on the playground.

I hope everyone on both sides of this realizes there is real value in having a separate repository (more flexibility, more ownership) and there is real value in merging them (less duplication in terms of infrastructure, more eyeballs, easier to find).

I'd suggest that we (maybe @martenson) create an issue describing the infrastructure that Dagobah requires and that this repository doesn't provide - and we discuss the technical details in a new issue.

Update: There are lots of non-technical details - I didn't mean to shut down the issue - but if training-material doesn't provide the technical infrastructure Dagobah requires then merging seems like not an option. We should resolve those first and then continue to have this conversation IMO.

[nsoranzo]

I'd caution against using the term "Galaxy Team" here

I surely acknowledge and thanks the Galaxy Team for the contributions to this repo, which have been massive! I should have said something like "part of/people from the Galaxy Team". But still, think about how this duplication of efforts can be perceived from the outside, especially when coming from core contributors.

I think we all have seen too many times how this duplications end up: one of the two repos will be become abandoned/unmaintained in a few years time. It would be nice to not repeat this common mistake.

[martenson]

@jmchilton my point regarding infrastructure was different from your understanding I am afraid. I wrote:

atm I do not think we are missing any infrastructure provided by the training-material repo

What I meant is that I do not see any infrastructure benefits dagobah gains by transferring into a different repo. What dagobah uses is very simple -> Python + Makefile.

My opinion is that the cost of moving dagobah over does outweight the benefits.

I also have other minor points I have listed here when writing this but I have deleted them to keep focus on the main point outlined above.

[natefoo]

One point here - it wasn't necessarily our intent to have a separate repo for "Galaxy Team" materials, Dagobah was started as a quick and dirty way to get materials made quickly for the Utah admin training, and only admin training materials are there.

Personally, I'd prefer to have all of the materials under a single style and build system to encourage as much contribution as possible. However, I don't know what that means in terms of difficulty in merging.

My opinion is that the cost of moving dagobah over does outweight the benefits.

@martenson can you outline these costs? I am not familiar enough to know what they are.

[jmchilton]

@martenson I see now - sorry for misreading that.

[bgruening]

Debating about costs is probably not useful in this context. I don't think anybody can estimate costs and if there is desire to merge both it will be probably a community effort and not @martenson doing it alone. I would like to figure out if all our training should be in one repo or if we want to keep it separate. If separate where do we draw the line and how do we communicate this to contributors and users.

I don't mind to have also technical aspect mentioned in this issue, I'm happy about all information that I can get so we can discuss this next week in detail during our hackathon.

[martenson]

@natefoo Alright, here is my shortlist of what dagobah repo currently has:

• different style and focus on delivery (logical and time-bound trainings schedule)
  • better paradigm of preserving past training materials with branch-per-delivery (that can also be tied to a Galaxy version)
• less technological hurdles (I am looking at you, Ruby)
• different (remarkjs vs revealjs) markdown syntax for JS slides framework
• newly it has automated jenkins build and deployment

All of the above added together equal the majority of what I called the cost.

Additionally we would increase the complexity of repo understanding and contribution for both of the repos. The cost of this is hard to estimate.

@bgruening The fact the transfer is community effort does not lower the costs. The community could have spent its effort on something more productive is my point.

[bgruening]

• different style and focus on delivery (logical and time-bound trainings schedule)
• better paradigm of preserving past training materials with branch-per-delivery (that can also be tied to a Galaxy version)

This branch-per-delivery did not worked so far. We tried this last week and the GAMe training can not be redone as tarballs are missing. We tried to fix this issue in the training-repo by getting DOIs for all content that is extern. Despite this, I do not see why we can not tag (and freeze) the repo here as well with the exact same result. (no cost)

• less technological hurdles (I am looking at you, Ruby)

Funny that this was initially developed by @bebatut on OSX ;) But yes, if this is an argument we are lokking forward to join the technical infrastructure from the hub, or what ever is best to generate PDFs and HTML out of markdown. I care mostly about content.

• different (remarkjs vs revealjs) markdown syntax for JS slides framework

We use remarkjs nowadays. We tried revealjs, did a hackathon, collected feedback and changed it. (no cost)

• newly it has automated jenkins build and deployment

Same (not really it's travis) here and since last year, you could have saved a lot of time (cost) ;)

All of the above added together equal the majority of what I called the cost.

Hopefully, I could share my view and that this "cost" is not that much, in contrast to the cost that people duplicate material in both repos.

Additionally we would increase the complexity of repo understanding and contribution for both of the repos. The cost of this is hard to estimate.

How? The Hub is complicated because we have everything inside? BioConda is complicated because we have r and python packages inside?

@bgruening The fact the transfer is community effort does not lower the costs. The community could have spent its effort on something more productive is my point.

For example by checking 2 or 3 repos for content before starting to write their own? Don't get me wrong if this is the consensus I'm ok with it. But then we need to define what counts as admin training, what is dev training and what is user training. What belongs where and communicate this very well ... please estimate this "cost" :)

[martenson]

This branch-per-delivery did not worked so far. We tried this last week and the GAMe training can not be redone as tarballs are missing.

The fact a resource was missing does not have much to do with the branch-oriented paradigm and does not prove it as a failure.

in contrast to the cost that people duplicate material in both repos.

For example by checking 2 or 3 repos for content before starting to write their own?

There is no duplication with some secret efforts that are being hidden. If I am going to invest a considerable amount of time into a tutorial, checking one more repo is not costly to me. When deciding on what to do the contributor should always check resources outside of this repo anyways.

The current training-materials are already duplicating multiple efforts (like toolshed description and tool installation versus hub toolshed description and installation or dev to production versus hub pages on production environment. And the new training materials were created after these well-known resources already existed and were linked through.

If duplication is always bad, which I do not think it necessarily is, then training-materials repo itself are not doing the best job in addressing this.

Nothing we created in dagobah was duplication of materials in this repository (or any other known to us). The admin corner is basically full of placeholders so there is not much to duplicate.

Same (not really it's travis) here and since last year, you could have saved a lot of time (cost) ;)

Not building and hosting all separate branches, but I agree the cost here is fairly low.

The Hub is complicated because we have everything inside?

No. Hub would be complicated if we moved the whole training-material repo in it.

But then we need to define what counts as admin training, what is dev training and what is user training.

I think this is fairly easy to be frank.

[dannon]

Might be worth thinking about an analog to other artifacts we create/curate as a community. Let's take tools. Tools that see the most development, collaboration, and oversight are all in tools-iuc, right? I wouldn't begrudge anyone for wanting to keep their own tool repository separate, as that's their choice as the owner of the content, but there are very clear benefits to a tool being in the central repository. tools-iuc overtook tools-devteam a long while ago and hasn't looked back.

My 2¢ -- the benefits to putting it all (to be clear: all the training content) under one roof dramatically outweigh any costs associated with doing so

Followup: Have been talking with @bgruening about aligning the build processes between this repository and galaxy-hub, and it's going to be a major topic at next week's hackathon. It seems an obvious win, and reduction in duplication of effort, to use the same build processes, content types, etc.

One great example from Bjoern was that he'd like per-PR temporary website builds for proofreading/etc., so that people don't need to run the whole build system locally to see what changes will look like. The hub would greatly benefit from this functionality, too. Aligning our disparate tech stacks and reusing functionality like this, reusing styling, etc., would be a great benefit to the whole ecosystem.

[martenson]

Tools that see the most development, collaboration, and oversight are all in tools-iuc, right?

This is not a true or fair statement I think. tools-iuc repository has the most activity because it has the most tools, developers, and is prominently positioned by the galaxyproject 'marketing'. It does not make all the tools there 'best quality' or having 'best oversight'. There are many tools outside of this repo that have these qualities. And there are surely tools in IUC repo that are sub-par.

[nsoranzo]

Closing this in favour of #135.