search

nordic-rse / nordic-rse.github.io

The community of Research Software Engineers in Nordic countries.
https://nordic-rse.org
MIT License
13 stars 15 forks source link

Anouncing november seminar #328

Closed samumantha closed 3 years ago

samumantha commented 3 years ago

related #323

frankier commented 3 years ago
samumantha commented 3 years ago

For the abstract there is no strict guidelines: As short as possible, as long as necessary. So that people know if the topic is of interest to them. We could have a longer version in the hackMD (link added in last commit) if you want. :)

frankier commented 3 years ago

I find it a lot easier to write to an actual limit. It might be helpful for the future too. How about 200 words? 150?

samumantha commented 3 years ago

Ok, how about taking what is written under https://nordic-rse.org/events/seminar-series/#august-2021-package-development-in-julia as guidelines for length?

frankier commented 3 years ago

In terms of word count or height on the page?

samumantha commented 3 years ago

This one has ~100 words, so maybe 100-150 is a good word count?

samumantha commented 3 years ago

But really, I do not think we need to be strict here :)

frankier commented 3 years ago

Okay I'll write it to 150 words. Thanks.

frankier commented 3 years ago

Here we go:

While SLURM itself provides tools for job orchestration like job arrays,  high level tools like Snakemake and Ray are cluster agnostic and can either make use of SLURM, or run a laptop. To Snakemake and Ray to run within Singularity, I present `singreqrun`, which works by requesting the host runs programs on behalf of the container container.

The talk doubles as an introduction to Snakemake an Ray. After some brief background on the main tools (Singularity, SLURM, Snakemake and Ray), we proceed to shell code-along to run the following examples:

 * Snakemake for heterogeneous (mixture of CPU and GPU nodes) video corpus processing + quick porting across HPC clusters
 * Snakemake for text corpus processing including using extra Singularity containers for utilities
 * Ray for hyperparameter search

I end the talk by opening for discussion. Is this a good approach? Can we improve upon it?

I also added both abstracts to the HackMD.

lucaferranti commented 3 years ago

Great thanks! If you want, you can also add the short abstract to the webpage teaser in this PR. I think this would then be ready to be merged?

edit: Ah sorry, just noticed you didn't open this PR, sorry for the noise :)

frankier commented 3 years ago

https://github.com/nordic-rse/nordic-rse.github.io/pull/333