search

pmodels / oshmpi

OpenSHMEM Implementation on MPI
https://pmodels.github.io/oshmpi-www/
Other
25 stars 14 forks source link

ZE support #106

Closed raffenet closed 3 years ago

raffenet commented 3 years ago

Initial support for Level Zero device memory support. TODO: add test case.

Refs #103.

raffenet commented 3 years ago

This should contain everything need to create spaces with ZE memkind. I have not yet written code to test reading/writing to those memory regions. I prefer to leave that to April, if it can wait until then.

raffenet commented 3 years ago

Oh, I need to fix non-ZE builds. Force push coming shortly...

minsii commented 3 years ago

@raffenet Thanks. The PR looks good to me assuming that manual test has been done. To add a basic write/read test for ze, can we simply reuse the space_int_put|amo test? I'd prefer to have at least checked basic functionalities.

minsii commented 3 years ago

test:oshmpi/mpich/builtin test:oshmpi/ompi/builtin

minsii commented 3 years ago

test:oshmpi/mpich/builtin test:oshmpi/ompi/builtin

raffenet commented 3 years ago

@raffenet Thanks. The PR looks good to me assuming that manual test has been done. To add a basic write/read test for ze, can we simply reuse the space_int_put|amo test? I'd prefer to have at least checked basic functionalities.

I can give it a shot. With ZE, unlike cudaMemcpy, we need to create a stream and enqueue the operations onto it then wait for them to complete. I can probably copy/paste what is in the MPICH test suite with a little bit of work.

raffenet commented 3 years ago

I can give it a shot. With ZE, unlike cudaMemcpy, we need to create a stream and enqueue the operations onto it then wait for them to complete. I can probably copy/paste what is in the MPICH test suite with a little bit of work.

Basically there is a bunch of initialization and setup work that needs to go into gpu_common.h or otherwise. Might take another day or 2.

minsii commented 3 years ago

The plan sounds good to me. Thanks! I will approve this PR and merge it, as it is for space creation. The RMA/AMO tests and potential fixes can be in with a separate PR.