search

CHTC / templates-GPUs

Template job submissions using GPUs in CHTC
MIT License
39 stars 11 forks source link

Examples for accessing the GPU via Docker #1

Closed sameerd closed 5 years ago

sameerd commented 5 years ago

Hi,

Here are a few examples for accessing the GPU via Docker. There are two 'fake' examples and one 'real' example.

  1. Hello_GPU Just a one line execute file to see if we can talk to the GPU
  2. tensorflow_python Matrix multiplication on the GPU vs CPU. This is copied and modified from StackOverflow.
  3. pytorch_python This is a 'real' example with sample data that runs a convolutional neural network to classify the MNIST digits dataset. This example is taken from the Official Pytorch examples and then modified. The datafile is 33MB in size and is included with this PR.

Please feel free to change or modify anything and keep only what you need.

Rgds

agitter commented 5 years ago

Thanks for all the contributions @sameerd! These examples were high on our internal priority list.

This repository is so new that we don't have a pull request review procedure in place. I wanted to let you know that one of us will give feedback so we can work on merging this.

sameerd commented 5 years ago

You are welcome @agitter!

Lauren requested some examples after she helped me get this running last week. I sent her a message and I assume she will review the PR some point.

lmichael107 commented 5 years ago

Yes, thank you, Sameer!

We'll review you contributions soon. Even if there are formatting changes (we haven't decided yet on what might be best), we'll probably merge and then work, ourselves, at any tweaks, but we'll let you know if we have any questions or change requests that you'd be best to address.

On Wed, Aug 28, 2019 at 6:29 PM Sameer D’Costa notifications@github.com wrote:

You are welcome @agitter https://github.com/agitter!

Lauren requested some examples after she helped me get this running last week. I sent her a message and I assume she will review the PR some point.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/CHTC/templates-GPUs/pull/1?email_source=notifications&email_token=AA42KRFYYVSJDM7M6MOUTS3QG4C4ZA5CNFSM4IRSV36KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5MYPCY#issuecomment-525961099, or mute the thread https://github.com/notifications/unsubscribe-auth/AA42KRA3U5E25WWHHHIHOGDQG4C4ZANCNFSM4IRSV36A .

lmichael107 commented 5 years ago

Tony, thank you for the review. I've got another CHTC team member (@iross) who will run through everything and provide input

Regarding your points:

  1. What license would you suggest? I'm most familiar with what Software Carpentry uses (Creative Commons: https://github.com/swcarpentry/shell-novice/blob/gh-pages/LICENSE.md), but am also familiar with the basics of Gnu and MIT (https://choosealicense.com/).
  2. Sounds like a good use of the Readme for each example. @sameerd, do have any requests from us in terms of how we attribute the initial materials to you, and opinions as a contributor relevant to licensing? (It's okay if not.)
  3. I also like the idea of having output, in a different folder, even with the additional maintenance.
sameerd commented 5 years ago

Lauren, I don't have any requests re attribution. However, with licensing, the more permissive for users the better.

Software Carpentry has a pretty good license with documentation licensed under Creative Commons and code under the MIT license.

Anyway, I updated the two examples I copied from the internet with their own licenses and added two more commits.

iross commented 5 years ago

Looks good to me! Thanks for putting this together. Everything is clear and works out of the box on CHTC.

re: example output, I think that it works well as it currently is. I expect that these examples will be fairly stable, so the overheads (both maintenance and data sizes) are pretty minimal.

lmichael107 commented 5 years ago

Agreed. I think we can state that example data needs to be sufficiently small to not over-burden the repo (or users of it) or require data availability tools outside of regular HTCondor file transfer, OR that examples use data that is publicly available (downloadable, perhaps on CHTC's 'squid' web server) to the job.

lmichael107 commented 5 years ago

Regarding licensing, let's go with Creative Commons? Sameer, do you want to review the Readme files you created, and attribute to yourself with something like "Materials for this example were initially developed and contributed by ()." ?

sameerd commented 5 years ago

Lauren, if you don't mind I don't particularly want to put my name on these examples. It didn't take much time to develop. The Github pull request and commit record is enough of attribution for me.

lmichael107 commented 5 years ago

I think that's fine by us. Tony, shall we merge?

Lauren

On Tue, Sep 10, 2019 at 12:14 PM Sameer D’Costa notifications@github.com wrote:

Lauren, if you don't mind I don't particularly want to put my name on these examples. It didn't take much time to develop. The Github pull request and commit record is enough of attribution for me.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/CHTC/templates-GPUs/pull/1?email_source=notifications&email_token=AA42KRDSQXCIO6DOJDG2EVLQI7IW5A5CNFSM4IRSV36KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6L26FQ#issuecomment-530034454, or mute the thread https://github.com/notifications/unsubscribe-auth/AA42KRH77BBCDC3FT4EGKPTQI7IW5ANCNFSM4IRSV36A .