GAT2021 Followup Overhaul

hexylena commented 3 years ago

GAT 2021 Followup

This PR contains a large number of bugfixes targeted at issues discovered during https://github.com/galaxyproject/training-material/issues/2274

Fixes #2274

Admin Training: Automated

This branch includes numerous fixes for the admin training topic, collected from GAT2021:Online, but it also starts the project of automating GAT.

We now use diffs throughout the admin training series
These can be pulled out and added together, to build the full admin training playbooks
This is automated in https://github.com/hexylena/git-gat

There is alpha quality code for building the completely automated videos (slides to video, but instead tutorial to videos). This needs more testing before it's productive, we're aiming for GAT 2022 for that.

hexylena commented 3 years ago

[ ] add mod_zip, thanks @mvdbeek @lldelisle https://docs.galaxyproject.org/en/latest/admin/nginx.html#creating-archives-with-mod-zip
[ ] check that the file sending we added actually works
[ ] maybe add https://docs.galaxyproject.org/en/latest/admin/nginx.html#receiving-files-with-nginx too? it's better for performance

hexylena commented 3 years ago

[ ] update tiaas role if not done already
[ ] add note about which ports need to be exposed for people not using our machines (Thanks @dickgroenenberg!)

hexylena commented 2 years ago

Update: There is a new Github Action which automatically rebuilds https://github.com/hexylena/git-gat on any commit to the admin topic. Neat. Now we don't have to worry about keeping that up to date.

hexylena commented 2 years ago

~Proposal~ Fixed Diffs for Copying

Instead of a fancy button that makes diff 'fixed' for copying one by one, how about we just "fix" them all, automatically. It won't be proper diff format anymore (but is anyone really using those directly and running patch?)

$("div.language-diff pre code .gi,div.language-diff pre code .gd").each((x, e) => { 
  var fixed = $(e).text().split('\n').map(q => { return q.slice(1) }).join('\n');
  $(e).text(fixed);
})

fixes all of the snippets:

before	after

do I want a button that does it? yes. But will this also work on short notice? Also yes.

hexylena commented 2 years ago

Ok, I think we've got most everything addressed from https://github.com/galaxyproject/training-material/issues/2274

There's a couple of remaining things in there that can be pushed to later, but it's time to start reviewing and testing, and getting this pr merged.

Please try and note which tutorials you think need to be re-recorded. By my estimate:

ansible-galaxy (we dropped a whole step)
pulsar (we use vault now)
singularity (this is the one that gave EVERYONE problems)
ephemeris (a number of tools changed)

Reviewers

There have been very few text changes in the tutorials, I would encourage focusing on the automatically extracted git-gat, does everything work there
Ignore the video scripts until the future.

Pending

[x] galaxy role update https://github.com/galaxyproject/ansible-galaxy/pull/132

hexylena commented 2 years ago

ansible-galaxy's tests are passing!!!!!!

hexylena commented 2 years ago

singularity's playbooks run (without a test case), also we hit https://github.com/sylabs/singularity/issues/68 so which fork do we use? I'm confused.

hexylena commented 2 years ago

tutorial	playbooks	tests
ansible-galaxy	:heavy_check_mark:	:heavy_check_mark:
singularity	:heavy_check_mark:	missing
tool management	n/a	missing
cvmfs	:heavy_check_mark:	added + passing
data libs	:heavy_check_mark:	:heavy_check_mark:
connect-to-compute	:heavy_check_mark:	missing
job-destinations	:heavy_check_mark:	missing
pulsar	:heavy_check_mark:	ish. (we test that pulsar is running OK)

but I did something dumb with the galaxy systemd, it doesn't behave nicely if you just want to e.g. set the env, rather than also setting that it should be mules. That's what I get for using dictionaries instead of just a bazillion variables.

hexylena commented 2 years ago

@natefoo @Slugger70 that's all I have energy for, hope y'all can get the ball across the line on this one.

@Slugger70 would you be up for re-recording singularity + pulsar (since we use vault now)?
@natefoo would you be up for recording the ansible-galaxy tuto?
@cat-bro since you've made a couple changes, do you want to re-record ephemeris? Is it worth doing?

(or any other mixture of people recording but yeah.)

cat-bro commented 2 years ago

@hexylena I'll be rerecording the ephemeris stuff A note on git-gat: I'm currently running it on one of the gat machines and I needed to update the cache before it was able to install packages. I'm not sure if this belongs in the playbook or the bootstrap. A note on the ephemeris tutorial: I'm currently failing to install things on my gat machine with bioblend.ConnectionError: HTTPSConnectionPool(host='gat-2.oz.training.galaxyproject.eu', port=443): Max retries exceeded with url: /api/tools?in_panel=True&key=45fc909559337090b1969c4abcb80e9c (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)'))), 0 attempts left:. I don't think this came up in January. I'll have a go at troubleshooting this later on today.

hexylena commented 2 years ago

@cat-bro amazing, thank you!

Weird about the cache update. I had to switch the machines to the Ubuntu repos in NZ again, the Aussie ones didn't work for me again and I don't know why.

Weird to see that error. Galaxy-wait worked after I added the updated LE fake root certs, I would have expected bioblend to, too.

Op vr 4 jun. 2021 06:51 schreef cat-bro @.***>:

@hexylena https://github.com/hexylena I'll be rerecording the ephemeris stuff A note on git-gat: I'm currently running it on one of the gat machines and I needed to update the cache before it was able to install packages. I'm not sure if this belongs in the playbook or the bootstrap. A note on the ephemeris tutorial: I'm currently failing to install things on my gat machine with bioblend.ConnectionError: HTTPSConnectionPool(host='gat-2.oz.training.galaxyproject.eu', port=443): Max retries exceeded with url: /api/tools?in_panel=True&key=45fc909559337090b1969c4abcb80e9c (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)'))), 0 attempts left:. I don't think this came up in January. I'll have a go at troubleshooting this later on today.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/galaxyproject/training-material/pull/2371#issuecomment-854356299, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADP7O2TFLET6CGS5F3HWKTTRBLVBANCNFSM4XUUJJ4Q .

cat-bro commented 2 years ago

Looking through the scrollback I can see that galaxy-wait did work perfectly during the run of .scripts/1-ansible-galaxy-run.sh but no longer does.

ubuntu@gat-2:~/git-gat$ galaxy-wait -g https://gat-2.oz.training.galaxyproject.eu -v
[00] Galaxy not up yet... HTTPSConnectionPool(host='gat-2.oz.training.galaxyproject.eu', port=443): Max retries exceeded with
[01] Galaxy not up yet... HTTPSConnectionPool(host='gat-2.oz.training.galaxyproject.eu', port=443): Max retries exceeded with

Galaxy is running and logs look fine

hexylena commented 2 years ago

@cat-bro fixed, something was wrong with the certificates but can't tell what. The LE fake roots x1 and x2 were there (one had an unexpected filename, .pem.crt, both should have been .crt) so I'm suspicious that they didn't get deployed properly. I re-ran that task from bootstrapping and it works now.

hexylena commented 2 years ago

I cannot explain it working initially, that's,, interesting.

cat-bro commented 2 years ago

@hexylena thanks for fixing! scripts/1 was run, followed by scripts/2, followed by my installing ephemeris in a virtual environment. Whatever went wrong happened sometime after the end of scripts/1. That .pem.crt file is a fake one that I added to try to fix it.

cat-bro commented 2 years ago

The problem seems to arise from installing ephemeris in a virtual environment when it is already installed globally. This time it's only broken within the virtual environment. The first time it broke globally as well, no idea why.

hexylena commented 2 years ago

Ahhhh that could make sense. It was installed globally as part of the testing process.

Should we just install it globally? skip the venv step then?

cat-bro commented 2 years ago

Would it be safer if git-gat had a venv for its requirements? If it's installed globally there's a chance that a student would create the ephemeris_venv and break it. The ephemeris_venv is also part of the data libraries tutorial

[updated] The machines already have global ephemeris because they have planemo installed as part of the bootstrap process, so it must have been there last time too but the venv was ok. I'll add a step to check whether ephemeris is already installed in the tut and skip the venv step.

Slugger70 commented 2 years ago

@natefoo @Slugger70 that's all I have energy for, hope y'all can get the ball across the line on this one.

@Slugger70 would you be up for re-recording singularity + pulsar (since we use vault now)?

@natefoo would you be up for recording the ansible-galaxy tuto?

@cat-bro since you've made a couple changes, do you want to re-record ephemeris? Is it worth doing?

(or any other mixture of people recording but yeah.)

Hey @hexylena - I will re-record singularity and pulsar as soon as this PR is merged.

hexylena commented 2 years ago

@Slugger70 approved so I'm going to merge this. We'll put everything else in a new PR(s) but should be easier to contribute once this huge chunk is merged.

martenson commented 2 years ago

Nice job @hexylena 🥇

galaxyproject / training-material

GAT2021 Followup Overhaul #2371

GAT 2021 Followup

Admin Training: Automated