search

carpentries-incubator / proposals

Open an issue in this repository to share Carpentries-style lessons and lesson ideas.
88 stars 6 forks source link

HPC lessons #13

Open tracykteal opened 6 years ago

tracykteal commented 6 years ago

There has been general interest in lessons for working with HPC resources. An upcoming BOF at Supercomputing 2017 is set to discuss ideas about curriculum and core competencies. There's also a session on Best Practices for HPC training We'll update this issue as there is more information.

If you're interested in HPC lessons and discussions related to them, also join the mailing list

danielskatz commented 6 years ago

There's also a BoF on High Performance Computing Education in US Data Science at SC17: Wed, 5:15 pm, room 703.

tracykteal commented 6 years ago

@symulation added that ComputeCanada has put together some HPC materials https://github.com/hpc-carpentry

with work from @jstaf

tracykteal commented 6 years ago

Website with information from the SC HPC Carpentry BOF https://hpc-uk.github.io/sc17-hpccarpentry-bof/

Overview of HPC Lesson Topics and URLs HPC Lesson topics and URLs ongoing work

Materials put together by @aturner-epcc @devbioinfoguy @ChristinaLK @chrisbpawsey @callaghanmt

r-barnes commented 6 years ago

This is of interest to me - where are things at?

ChristinaLK commented 6 years ago

Hi Richard -- I saw your post on hpc-discuss as well. There's work happening in two places:

The materials in the first link are being developed for a specific site and we're still figuring out if we want to add community material there, or port them to a different location. @psteinb has also been hacking on his own version of materials at https://psteinb.github.io/hpc-novice/ My current goal is to figure out which location we want to use for community development + bring together the two branches of material into that place.

What are you interested in contributing / working on? Depending what you want to do, there's probably some immediate contributions to be made. Let me know!

This is a good incentive for me to get out the blog post I've been sitting on about this...

r-barnes commented 6 years ago

@ChristinaLK: I develop Python HPC workflows and help mentor Oak Ridge's OpenACC/OpenMP hackathons.

I'll shortly be publishing a paper that walks through the development and profiling of a simple landscape evolution model. That material seems as though it could be recycled for pedagogy (though it's in C++: I'm not sure if that's one of your target languages).

If you have a list of spots that need work, I could look into tackling one.

Also, I notice that the current lesson doesn't seem to include anything involving the peculiarities of filesystems (like Lustre).

psteinb commented 6 years ago

Hi to all,

my material of hpc-in-a-day is consolidating rapidly driven by the feedback people gave me. As I mentioned elsewhere, I am happy to contribute it into https://github.com/swcarpentry/hpc-novice. I simply have to find the time. ;) If anyone wants to help out, please give me a shout.

I'll be coming to CarpentryCon in Dublin and the agenda already lists quite some slots for HPCCarpentry - which I hope could be a venue to agree on a direction and possibly start filling lessons with content.

@ChristinaLK I was surprised to find out about https://github.com/hpc-carpentry/hpc-intro and the recent activity therein. I see tremendous overlap with the content of hpc-in-a-day. I personally agree very strongly that we have to unite efforts and not develop the same thing in 2 places. Working with the shell, ssh'ing and using the batch system is something all tutorials have in common.

@r-barnes Please send around the paper you mentioned. I am interested.

Best, Peter

aturner-epcc commented 6 years ago

@r-barnes There are lots of places that need work. As @ChristinaLK mentions, there is a bit of a decision to be made on which current work we work off. In my opinion, the work @psteinb has been doing is closest to what the community that attended the BoF seem to want.

You mention C++. To clarify, the initial HPC Carpentry that we're currently working on, is an introductory course so, to a first approximation, no languages will be covered (although how to use the application development environment, e.g. TCL/LMOD modules probably will be discussed) . Take a look at the birds-of-a-feather session from SC17 to get an idea of the learner profile:

https://hpc-uk.github.io/sc17-hpccarpentry-bof/

In particular, the section on learner profile in the BoF report:

https://hpc-uk.github.io/sc17-hpccarpentry-bof/pdf/sc17-hpccarpentry-bof-report.pdf

This does not mean that there will not be more advanced HPC Carpentry courses which are aimed more at the programming community - I think there would be a lot of interest in this.

psteinb commented 6 years ago

@rturner-epcc what do you mean by:

On 02/08/2018 09:11 AM, aturner-epcc wrote:

However, the work at https://github.com/hpc-carpentry/hpc-intro has a real implementation already.

in comparison to https://psteinb.github.io/hpc-in-a-day/ ?

aturner-epcc commented 6 years ago

@psteinb Apologies! This was a mistake - I thought I was looking at your repo and was looking at https://github.com/swcarpentry/hpc-novice by mistake and thought the implementation was not in place yet. I see that it is - must double-check URLs before posting!

I have edited the comment to fix this.

psteinb commented 6 years ago

thanks for editing your post.

aturner-epcc commented 6 years ago

@psteinb What is the best way to hpc-novice contribute at the moment? Should I fork https://github.com/psteinb/hpc-in-a-day (which is where the most current material is) or the base https://github.com/swcarpentry/hpc-novice and contribute there (and it will catch up). Or, should I do something more complex and fork hpc-novice, add hpc-in-a-day as a read-only remote, work on my own branch and issue PRs back to hpc-novice?

Do you have an opinion on how you would like this done?

psteinb commented 6 years ago

Personally speaking (I invite all parties to provide feedback to this), I would love to extract parts of hpc-in-a-day and put them via dedicated PRs into swcarpentry/hpc-novice. This also holds for any other material you can think of from my side. But this is my isolated view on the world ;) as I am not clear which role https://github.com/hpc-carpentry/hpc-intro can/must/should play.

aturner-epcc commented 6 years ago

@psteinb Thanks. OK, my strategy for the moment will be to issue PRs to https://github.com/swcarpentry/hpc-novice in places where it does not impact work you have already done. If I have ideas to add to your stuff (not planned at the moment) then I will probably issue PRS directly to https://github.com/psteinb/hpc-in-a-day

jstaf commented 6 years ago

Hi all, wanted to drop in and update people on where the HPC Carpentry lessons are at (https://github.com/hpc-carpentry/). A year ago, we (Compute Canada) sat down and took a survey of how HPC was being taught at our sites and what teaching materials were available - we ended up deciding to sit down and write some new materials. Currently, we have 3 complete and relatively polished lessons (I also made a nifty landing page):

At this point, we are moving towards a wider release, and have already merged a number of community pull requests. The end goal here is to release a set of "Software Carpentry" materials that anyone can use to teach HPC. Contributions, pull requests, and criticisms are welcome.

If you are looking to contribute, there are a few areas we could still use some help:

ErinBecker commented 5 years ago

These materials were listed on the DEPRECATED sharing-lessons Etherpad. Some may be duplicates of resources already shared on this thread:

ErinBecker commented 5 years ago

@hpc-carpentry, @hpc-uk, @rantahar, @psteinb - We've just created The Carpentries Incubator GH organization to provide a single point of entry for community members to share lesson materials and discuss ideas for new lessons. Your lesson materials were previously listed on the sharing-lessons Etherpad and/or the lesson-ideas repo, which have now been deprecated. All of the resources listed there have been migrated here. We're still working on organizing this repo, so please do let me know if you have ideas for how best to organize this information to make it accessible to the community.

psteinb commented 5 years ago

Hey @ErinBecker I'd be very interested in putting hpc-intro and friends into a carpentries incubator state. @ocaisa I just moved jobs, so am a bit in leaving/arriving stage. Is there any template I could have a look at so that I can send it around and discuss the matter with other stakeholders?

ErinBecker commented 5 years ago

Congrats on the new job @psteinb! The Incubator is appropriate for lesson materials at any stage of development, so there's no need to do any more work on the repo in order to have it hosted here. The only requirements are that:

If you can confirm the last two requirements, I'd be happy to facilitate adding these lessons to the Incubator. Please let me know if you'd like to move ahead and I can give you the appropriate access. No rush!

ErinBecker commented 5 years ago

@psteinb @ocaisa - please let me know if there's anything I can do to help with transferring your lessons into the Incubator. No rush, but I know these things aren't always straightforward from the GitHub interface side and I'm happy to help where useful!