Is Level0 Virtual memory API supported?

[mgrabban]

Hello, I was wondering if virtual memory API here are supported because I am getting errors when trying to use some of the functions there.

I took this main.cc, added the following function and called it from inside main(). I built it following the build instructions here. It compiles and runs, but gives error.

void ReserveVirtualAddressSpace()
{
    ze_result_t status = ZE_RESULT_SUCCESS;

    std::vector<ze_driver_handle_t> driver_list = utils::ze::GetDriverList();
    PTI_ASSERT(driver_list.size() != 0);
    PTI_ASSERT(driver_list[0] != nullptr);

    ze_context_handle_t hContext = utils::ze::GetContext(driver_list[0]);
    ze_device_handle_t  hDevice  = utils::ze::GetGpuDevice();

    // Query page size for our 1MB allocation.
    size_t pageSize;
    size_t allocationSize = 1048576; // 1MB
    status = zeVirtualMemQueryPageSize(hContext, hDevice, allocationSize, &pageSize);
    PTI_ASSERT(status == ZE_RESULT_SUCCESS);
    std::cout << "pageSize: " << pageSize << std::endl;

    // Reserve 1MB of virtual address space.
    size_t reserveSize = (allocationSize + pageSize - 1) / pageSize * pageSize;
    std::cout << "reserveSize: " << reserveSize << std::endl;
    void* ptr = nullptr;
    status = zeVirtualMemReserve(hContext, nullptr, reserveSize, &ptr);
    PTI_ASSERT(status == ZE_RESULT_SUCCESS);

    // // Create 1MB physical memory object
    // ze_physical_mem_handle_t hPhysicalAlloc;
    // size_t physicalSize = (allocationSize + pageSize - 1) / pageSize * pageSize;
    // ze_physical_mem_desc_t pmemDesc = {
    //     ZE_STRUCTURE_TYPE_PHYSICAL_MEM_DESC,
    //     nullptr,
    //     0, // flags
    //     physicalSize // size
    // };
    // status = zePhysicalMemCreate(hContext, hDevice, &pmemDesc, &hPhysicalAlloc);
    // PTI_ASSERT(status == ZE_RESULT_SUCCESS);

    std::cout << "Success!\n";
}

ze_vmm/build $ SYCL_DEVICE_FILTER=lext_oneapi_level_zero:gpu:0 make && ./ze_vmm -l
Scanning dependencies of target ze_vmm
[ 50%] Building CXX object CMakeFiles/ze_vmm.dir/main.cc.o
[100%] Linking CXX executable ze_vmm
[100%] Built target ze_vmm
Driver #0: API Version 1.3
-- Device #0: Intel(R) Graphics [0x0bd5]
---- Subdevice #0: Intel(R) Graphics [0x0bd5]
---- Subdevice #1: Intel(R) Graphics [0x0bd5]
-- Device #1: Intel(R) Graphics [0x0bd5]
---- Subdevice #0: Intel(R) Graphics [0x0bd5]
---- Subdevice #1: Intel(R) Graphics [0x0bd5]
pageSize: 65536
reserveSize: 65536
ze_vmm: /nfs/pdx/home/mgrabban/intel/pti-gpu/samples/ze_vmm/main.cc:353: void ReserveVirtualAddressSpace(): Assertion `status == ZE_RESULT_SUCCESS' failed.
Aborted (core dumped)

What I notice is this

• zeVirtualMemQueryPageSize() returns ZE_RESULT_SUCCESS and I get a minimum pageSize of 65536. So I assume virtual memory API is supported.
• However, both zeVirtualMemReserve() and zePhysicalMemCreate() fail the asserts above.

Questions:

• Are the virtual memory functions supported? (I know SYCL does not support them yet).
• If they are supported, am I using them correctly?

I am trying to replicate CUDA driver API calls for virtual memory management in Level Zero.

Thanks!

[jandres742]

@mgrabban thanks. I guess you are referring to L0 GPU driver (https://github.com/intel/compute-runtime). Those APIs are not yet supported. Support is currently work in progress.

[mgrabban]

@jandres742 thanks for responding. Yes you are right. Do you want me to open an issue on https://github.com/intel/compute-runtime or is having it here fine? Also do you have any time line regarding when those APIs are going to be usable? How does one also push for prioritization? :-) I was hoping to migrate tiny-cuda-nn workload by NVlabs into SYCL but it makes extensive use of virtual memory API listed here and we do not have support in SCYL for them. I was asked to find out if Level Zero had support for them so that they could be included in SYCL as well.

[jandres742]

thanks @mgrabban . Ok, we will follow up with SYCL team to provide the support they need to implement this.

[jandres742]

@bmyates : please close this, we will follow up internally with sycl team.