search

microsoft / DirectML

DirectML is a high-performance, hardware-accelerated DirectX 12 library for machine learning. DirectML provides GPU acceleration for common machine learning tasks across a broad range of supported hardware and drivers, including all DirectX 12-capable GPUs from vendors such as AMD, Intel, NVIDIA, and Qualcomm.
MIT License
2.24k stars 299 forks source link

For NPU Sample, add flexibility in device creation options and fix Generic ML Device logic #624

Closed Zu-shi closed 1 month ago

Zu-shi commented 3 months ago

Expand NPU sample's capabilities for creating devices based on attributes.

More specifically, allow options to filter based on allowed, unallowed, and required attributes. Then add some flags for the most commonly needed options for this sample.

The logic for how each interacts with CreateAdapterList is a bit unconventional since CreateAdapterList ANDs the passed in attributes.

Tested locally on Intel NPU.

hez2010 commented 2 months ago

Can this run on NPU of Snapdragon X series as well?

Zu-shi commented 1 month ago

Can this run on NPU of Snapdragon X series as well?

Hi, I don't have a device on hand, but this is expected to work on Snapdragon X. If an issue is encountered during inference, the mobilenet version from QualComm AI Hub should work.

Zu-shi commented 1 month ago

I think this sample should be kept as simple as possible and only show logic to unambiguously select an NPU. I don't think we need to illustrate the flexibility and numerous ways adapters can be filtered (this isn't a DXCore sample); I think this flexibility actually makes the sample more confusing, because we're not guiding users on the precise attributes required to leverage NPU.

In short, I suggest having something like GENERIC_ML || (CORE_COMPUTE && !GRAPHICS).

A bit delayed but this has been incorporated, thanks!