WIP: Tool integration tutorial

[lecorguille]

@bebatut

• First review of Berenice slides
• Some reorganisation in upper sections
• Some modifications : Some add
• A List of TODO

I hope I didn't completly put the mess or upset you with my modifications 🙏

I think that it should be good to split the Planemo part "Integration" and "Publishing".

TODO (according to me):

• [x] configfiles
• [x] $__app__
• [x] macro
• [x] dataset collection
• [ ] html output
• [x] tool_data_table_conf.xml
• [x] \${GAALXY_SLOTS:-1}
• [ ] Datatypes
• [x] GitHub -> TravisCI

I try to add some <aside > but I didn't manage to see them!

I will now working on a conda build tutorial/hands_on in markdown... Expect if you have already something?

[bgruening]

A few comments:

• Dev-Corner/images/galaxy_logo.png and the general Galaxy images should be shared this is not Galaxy Admin specific and @shiltemann had also some similar images in her training. So maybe you can reuse them.
• Dev-Corner/images/planemo-logo.png shared?
• I would completely remove all occurrences from tool_dependencies.xml

I like the overall introduction a lot and think this is very valuable. The last parts of the slides are a little bit too far fetched for my opinion - especially if you want to write about configfiles etc. imho this should be covered by planemo.

[lecorguille]

@bgruening

Dev-Corner/images/galaxy_logo.png ... oups! Next commit 👍

The last parts of the slides are a little bit too far fetched for my opinion Can you give me a range of slide?

[bgruening]

More or less from ### citations on. I think this is all pretty well covered by the planemo tutorial, isn't it. I can imagine that you introduce input, outputs, help shortly in slides and than move on to planemo but explaining all things ... imho we should move as fast as possible to planemo and the new XSD documentation as primary resource.

Let me know what you think - and as usual thanks a lot for your contribution!!!

[lecorguille]

Some TODO done. But if any of you think that a part need to be completed or on the contrary need to be removed, don't hesitate.

[lecorguille]

ping @bgruening @jmchilton @bebatut

Late last night, I didn't see your comment Björn.

I wanted to merge quickly and durtily our 2 materials @bebatut and mine. It's rather done.

But, now, I guess I went too far for your standart ... again ... :)

Hum..., I'm used to increase the difficulty gradually. The Planemo tutorial starts with conda, planemo tool_init, for me, it's a little bit dense to replace a simple tag presentation. And it's the same for the XSD presentation, too complete for beginners.

So the challenge for me is to propose something in which I (and maybe other trainers) can be confortable with, convenient and not redondante or in compete with Planemo and Conda docs.

Note that it's not the contain that I'm speaking about but the form which don't suit me in front of usually uses: documentation vs course

It's the same for hands-on. I'm used to give a goal and let to the attendees one hour to solve it. The Planemo examples show to quickly the answers.

Note that I'm ready to move :)

Maybe the material is depending of the course format (duration, balance theory/pactrice) and of the audience?

[lecorguille]

We have just finish a Hangout with @bebatut I will reorganize/modify the slides so no need to review before the next commit.

From then, we will have a good base to discuse about.

[lecorguille]

@bebatut I need a review if you have time @abretaud Thanks for your picture about miniconda and maybe some other stuffs I plan to integrate. If you have time too, can you look at those slides. We will need to present them next week 👍

[bgruening]

I will give you a proper review tomorrow if this is ok. Thanks for your work!

[bebatut]

I am reviewing it

[lecorguille]

Thanks @bebatut! See you next commit!

[lecorguille]

Thanks @bebatut @abretaud A minor commit

So now, I will focus on contents. I postpone the image and code resize.

[lecorguille]

@abretaud can you propose something about Dataset collection in a light mode which can be completed by the planemo docs Maybe diagrams and/or screenshots and a word about <discover_datasets

[abretaud]

Ok, I'll try to do something today

[bgruening]

There are two 4. in Dev-Corner/images/big_picture.png

[bgruening]

Any thoughts about adding the docker and ansible images into the shared folder. I can imaging the admin section will reuse them as well. This looks great to me!!

[lecorguille]

I still have to set the conda hands-on. I plan de propose to build a conda package for 2 simple softwares: classic binary with a make; make install and a R package skeleton. Is anyone have some suggestions?

[bgruening]

vcftools should be known and simple enough: https://github.com/bioconda/bioconda-recipes/blob/f5eb63e30a76fd13c28663786d219c9f7750267c/recipes/vcftools/0.1.11/meta.yaml

I would take python as scripting example, R can produce many problems. Simple json has no dependency and should be easy as well.

[lecorguille]

@bgruening I choose seqtk :) The first training session will take place tomorrow, depending of the feedbacks, we might change to vcftools

About that, although, maybe this evening, I will apply some minor changes (definitely a lack of time the last week and this weekend to finalize). Can you ( @bgruening @bebatut ) imagine merging all this stuff on the master?

Thus, the next PR will be from the training feedbacks.

[lecorguille]

I forgot to mention that merging the branch will allow to access to the slides using this URL: http://bgruening.github.io/training-material/Dev-Corner/slides/tool_integration.html#/ That way, we will be able to share the slide with the attendees.

[bgruening]

@lecorguille sorry was travelling. Let's merge it and work on it further from the master branch. I'm really interested in your results and first feedback! Let us know!

[lecorguille]

@bgruening Cool, thanks. It was time to close this PR, it took too long for GitHub to display it.

Finally, I will not be trainer in this course for personal reasons. But @abretaud @loraine-gueguen should propose feedbacks.

[abretaud]

Course (almost) finished, time for feedback!

It went quite well, people managed to begin writing their own tool and seemed to understand well the process

A few things that could be improved:

• most of the people didn't know git/github, so I had to to show them how to create a pull request. maybe this could be added?
• a heavy silence fell on the room during the conda packaging hands-on :smile: people were lost and didn't know how to start. So describing a little more the meta.yaml/build.sh would help I think
• after the course students had 1 day to wrap their own tool. We had the case of a R script that needed many R libs not yet available in conda, they had to switch to another tool otherwise they would have spend the whole day packaging R libs. No miracle solution for this I think...

Other trainers will maybe complete this (and we'll have some feedback from students also)

[lecorguille]

Sorry again for not having been present :) So Cool!

About your second point, I agree but during this exercise (my "Galaxy Training Material Experience"), I found that it was hard to set the good balance point between be enough complete and not compete existing documentations (planemo, conda)

@bgruening @bebatut It should be interesting to keep tracks (dates, commit number, trainers, number of attendees, real duration, feedbacks, ...) of the course given with those materials

[loraine-gueguen]

Yes, it went well! Stuffs that can be added to improve the slides, to my mind:

• example for macros
• generate an html file as output (multiple output)
• some pictures are too small

[loraine-gueguen]

And one more thing:

What about hands-on answers ? For the planemo-toolshed course, we put answers on slides (last slides). But afterwards, I think this is not a good idea because trainees couldn't help to look at the answers during hands-on. Maybe the slides with the answers are not necessary and it would be sufficient to show how to do directly on the screen, once the hands-on time has elapsed. And the planemo commands are already detailed in the course.

[bebatut]

@lecorguille Good idea. We will think about a way to keep such track. Any suggestions?

@abretaud @loraine-gueguen Thanks for the feedbacks. I will keep a track to try to improve this material

@loraine-gueguen The answers is still an open question. We do not have a good solution. Maybe in hands on text tutorial, we can find a way to include them.

@lecorguille Do you think we can have also the tutorial in markdown? It can be used for self training but also for workshop. What do you think?

[lecorguille]

@bebatut Sure but without real deadline like we had with this training, I will have to kick my own 4$$ to find time :)

[shiltemann]

@abretaud @bebatut Maybe we could make a few micro-tutorials for some basics like how to work with git(hub), and maybe things like Trackster or how to work with collections etc, I can imagine these things will be used in multiple tutorials, and would be nice if you could just add a link like "if you are not familiar with X, please look here" instead of explaining the same thing in multiple tutorials.

@lecorguille @bebatut yes! @ keeping track of trainer experiences, would be handy to look through when preparing to teach, and maybe can keep track of FAQs by students and trainers can discuss answers to these (sometimes I teach a course where biology of it is bit out of my comfort zone, nice to be able to anticipate some questions etc) ..maybe a instructor-discussion issue per tutorial? and/or start a github wiki for this repo?

[abretaud]

+1 for microtutorials! And for FAQ/instructor guide too, but not sure on the form of it