neuroinformatics-unit / HowTo

NIU website on common software problems and their troubleshooting
http://howto.neuroinformatics.dev/
Creative Commons Attribution 4.0 International
9 stars 1 forks source link

How to use the SLEAP HPC module with SLURM #40

Closed niksirbi closed 8 months ago

niksirbi commented 10 months ago

This PR adds two how-to guides:

  1. A very long and (hopefully) comprehensive guide on how to use the HPC SLEAP module for remote training and inference. An earlier version of this guide existed in a separate repo and had been reviewed by @sfmig and @lochhh. I've incorporated much of the feedback, and added more structure to the guide, with dropdowns, troubleshooting sections etc. I've also made sure that it's up to date with the latest version of the SLEAP module.
  2. A "SLURM arguments primer", that explains the most commonly used SLURM directives, and what values they should take. SLURM has its own documentation, but it's quite extensive and it's hard to find things in there (especially for non-technical people). So I thought making a shorter list of the type "the n most useful arguments" would be of value.

Other than that, there are some small bits and pieces, like:

I would suggest @lochhh looks at the SLEAP part, because she has seen an earlier version of it already, and @adamltyson reviews the SLURM bit, as it might be useful/relevant for the upcoming HPC course.

The best way to review this is to directly look at the rendered website build. I've temporarily overridden the workflow rules to allow publication from this PR, which means that I have to:

adamltyson commented 8 months ago

Out of interest, why this way, rather than the reviewers building locally

Just my neglect. I started working on this very long ago, when not all of us were super familiar with sphinx local builds and I intended to make life easier for reviewers.

But now that we are all such Sphinx pros, there is no need to do that, and I should stop doing it. I had just forgotten that it was configured "the old way" on this branch, and I only realised after the build went public 😨

adamltyson commented 8 months ago

@niksirbi the bigger problem is the continued insane GitHub bug that ended up with you updating my comment!?!

niksirbi commented 8 months ago

@niksirbi the bigger problem is the continued insane GitHub bug that ended up with you updating my comment!?!

exactly, it almost happened again, but I realised it in time now

adamltyson commented 8 months ago

@niksirbi I also don't get why we can edit each others comments anyway. I can use the dropdown to edit your messages using the GitHub UI. Is it because of the permissions we have for the repo?

niksirbi commented 8 months ago

@niksirbi I also don't get why we can edit each others comments anyway. I can use the dropdown to edit your messages using the GitHub UI. Is it because of the permissions we have for the repo?

Maybe because we are all admins. Should I set neuroinformatics-all as maintainers?

adamltyson commented 8 months ago

Maybe because we are all admins. Should I set neuroinformatics-all as maintainers?

yeah that sounds good. Especially as this is the new, larger set of contributors.

niksirbi commented 8 months ago

Maybe because we are all admins. Should I set neuroinformatics-all as maintainers?

yeah that sounds good. Especially as this is the new, larger set of contributors.

Done, do you still have permission to edit my posts (you shouldn't now)?

adamltyson commented 8 months ago

Done, do you still have permission to edit my posts (you shouldn't now)?

I do, but I'm admin of the whole org, so I guess that's expected (but still weird).

niksirbi commented 8 months ago

Done, do you still have permission to edit my posts (you shouldn't now)?

I do, but I'm admin of the whole org, so I guess that's expected (but still weird).

FYI https://github.com/isaacs/github/issues/266

adamltyson commented 8 months ago

Crazy. Anyway, yell at me if I accidentality edit any comments and I'll try to be more careful. I really don't think we should be editing what other people say!