search

KhronosGroup / OpenCL-SDK

OpenCL SDK
Apache License 2.0
577 stars 120 forks source link

Add Python samples? #30

Closed inducer closed 3 years ago

inducer commented 3 years ago

At a recent CL advisory panel meeting, there was discussion about adding Python samples to the SDK. The ones I've currently collected live here:

https://github.com/inducer/pyopencl/tree/main/examples

I would say that these are of varying quality and scope. The most interesting ones are potentially these:

Simple functionality demos

App-level demos

These represent more complete programs that were contributed by @numa65536 in https://github.com/inducer/pyopencl/pull/436. Maybe he'd be open to contributing/maintaining them here and/or adding perspective.

One concern I have about adding these examples here is CI. In the PyOpenCL repo, there is CI to ensure that these examples continue to work. If they were copied/moved here, it would be good to have a similar setup to ensure nothing breaks. The corresponding CI job is not complicated.

MathiasMagnus commented 3 years ago

One concern I have about adding these examples here is CI. In the PyOpenCL repo, there is CI to ensure that these examples continue to work.

Then we agree. If something doesn't have CI, from a practical standpoint it doesn't exist. We are aiming to have extensive samples coverage with both kinds of tests: simple and app-level. Even if the source samples are of varying quality and scope, having them as inspiration to tap into is much appreciated.

inducer commented 3 years ago

Great to hear. What would the next steps be? File a PR here that adds these and a CI job?

numa65536 commented 3 years ago

Hello,

I will try, as I can, to make my examples working on this new approach. I will do my best. But from my point of view, after 10 years of real GPU programming, clearly, PyOpenCL is and stays the most versatile way to implement code on parallel devices... Now, I can instantly run my codes on more than 50 different CPU and more than 70 GPU directly available il my IT test center ( http://www.cbp.ens-lyon.fr/python/forms/CloudCBP ). My examples are a part of a larger project : bench4gpu initiated more than 10 years ago : https://forge.cbp.ens-lyon.fr/redmine/projects/bench4gpu/repository

Best regards.

EQ

inducer commented 3 years ago

@numa65536 Thanks for your response, and for your willingness to help. Also, I should have explained the context better. (Sorry!) This is not about a new approach at all. To my mind, the purpose of this SDK is simply to have a collection of examples of OpenCL in use, similar to what some vendors provide for their APIs. The idea is to help potential users get started by being able to draw inspiration from existing examples.

In this case, my view is that we would simply include some examples of the use of PyOpenCL here in order to give potential users a glimpse of that "way of doing business". From your end, this shouldn't look any different from having the same code live in PyOpenCL's examples.

inducer commented 3 years ago

Before I go open a PR for this: what would a good path for these examples be? The existing structure appears to be very CMake-centric. Is it OK if I just make a python/core folder at the top level and add things there?

MathiasMagnus commented 3 years ago

@inducer I believe opening a PR is adequate, as that's where the nitpicking happens (in all repos). Having Python samples is a wanted feature, that's been established. Getting the basics right will be done through code review. IMHO python/core seems appropriate.

Since Python samples don't require building in the same sense C/C++ samples do, building doesn't need to be hooked up to CMake. However, if execution will be tested (even if not result verification, just checking that they don't crash), I would be against involving any other testing framework which aren't present in other OpenCL repos (mainly CTest) where they can't get the job done.

Kerilk commented 3 years ago

Thanks for submitting python samples @inducer and @numa65536 !

I plan on submitting Ruby samples and am prototyping a pull request here: https://github.com/Kerilk/OpenCL-SDK/pull/1 I think we need at least a samples/bindings directory or a top level bindings directory.

I did some CI on Ubuntu with pocl, and a few with MacOS, but apparently they broke their CPU Opencl in the latest security patch from the 28 of April, and building programs fails... Anyway for now I built dedicated CI runs for ruby. I think I can come up with a matrix that would cover all ruby CI tests in the end.

inducer commented 3 years ago

I think we need at least a samples/bindings directory or a top level bindings directory.

Personally, I think I'd prefer exposing all the bindings at the top level, with python, ruby as individual directories. If they're hidden under bindings, it just makes them less discoverable.

inducer commented 3 years ago

I did some CI on Ubuntu with pocl, and a few with MacOS, but apparently they broke their CPU Opencl in the latest security patch from the 28 of April, and building programs fails... Anyway for now I built dedicated CI runs for ruby. I think I can come up with a matrix that would cover all ruby CI tests in the end.

I am hoping to use https://github.com/conda-forge/ for CI. It's three-odd commands to get a working CL install. Are the ruby CL bindings packaged for conda-forge?

Kerilk commented 3 years ago

I think we need at least a samples/bindings directory or a top level bindings directory.

Personally, I think I'd prefer exposing all the bindings at the top level, with python, ruby as individual directories. If they're hidden under bindings, it just makes them less discoverable.

I was worried about cluttering the base folder, but you are probably right.

I am hoping to use https://github.com/conda-forge/ for CI. It's three-odd commands to get a working CL install. Are the ruby CL bindings packaged for conda-forge?

I install them using gem: https://github.com/Kerilk/OpenCL-SDK/blob/0cf14269eec269a3cbb0de18844bd271c5034499/.github/workflows/presubmit.yml#L68-L69 ruby's own installer, but I am open to other mode of distribution.

numa65536 commented 3 years ago

So, if I understand clearly your message, you just want me to put my examples I provided in examples of PyOpenCL in https://github.com/Kerilk/OpenCL-SDK/tree/0cf14269eec269a3cbb0de18844bd271c5034499/samples in order to give use cases of PyOpenCL. I suppose it would be better if I introduce some explanations about the context, the use and the results of the 3 examples I provided. For example, would you like some explanations like these ones ?

For these 3 examples, the initial goal was to compare performances (so metrology included) and portabilty for all OpenCL devices and implementations.

In fact, I oftenly use these 3 codes as "Matrix" codes : is this that your have in mind ? Showing that sample codes can be used to start a project (using perturbation method).

If you confirm that I have understood your wishes correctly, I will put my examples is a specific folder, for example, samples/python/reallife or samples/python/metrology

Le mer. 12 mai 2021 à 19:23, Andreas Klöckner @.***> a écrit :

@numa65536 https://github.com/numa65536 Thanks for your response, and for your willingness to help. Also, I should have explained the context better. (Sorry!) This is not about a new approach at all. To my mind, the purpose of this SDK is simply to have a collection of examples of OpenCL in use, similar to what some vendors provide for their APIs. The idea is to help potential users get started by being able to draw inspiration from existing examples.

In this case, my view is that we would simply include some examples of the use of PyOpenCL here in order to give potential users a glimpse of that "way of doing business". From your end, this shouldn't look any different from having the same code live in PyOpenCL's examples.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/KhronosGroup/OpenCL-SDK/issues/30#issuecomment-839956798, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA2Y5JNVXGR25E66H3S3JILTNK2QLANCNFSM44R2PHUQ .

Kerilk commented 3 years ago

So, if I understand clearly your message, you just want me to put my examples I provided in examples of PyOpenCL in https://github.com/Kerilk/OpenCL-SDK/tree/0cf14269eec269a3cbb0de18844bd271c5034499/samples

No I don't think so, I just wanted to open the discussion so that we would create pull requests with some coherency. But I see @inducer has open one already: https://github.com/KhronosGroup/OpenCL-SDK/pull/31 I will refactor mine to match the same architecture.

inducer commented 3 years ago

I think #31 is ready for a first look. No idea if the CI passes since it won't run without approval. I've left @numa65536's app-level examples out for now to avoid complicating the copyright assignment process. Maybe he can add these later (perhaps in the versions from https://github.com/inducer/pyopencl/pull/470)

bashbaug commented 3 years ago

Since we've merged the first round of python samples (see #31) can we close this issue?