search

furiosa-ai / device-api

APIs that offers NPU devices' information and allow to control the devices
Apache License 2.0
5 stars 8 forks source link

`find_devices` should consider fragmentation #29

Open sukyoungjeong-furiosa opened 2 years ago

sukyoungjeong-furiosa commented 2 years ago

Let's assume 2 warboy NPUs (namely, npu0 and npu1) with a core of npu1 occupied.

(npu0) {0: Available, 1: Available}
(npu1) {0: Occupied("test"), 1: Available}

If someone is looking for an available singlecore devfile, it would be pleasant if find_devices(..) returns npu1pe1.

But current implementation returns npu0pe0 since it does not consider fragmentation.

[DeviceFile { device_index: 0, core_indices: [0], path: "/root/src/device-api/test_data/test-0/dev/npu0pe0", mode: Single }]
sukyoungjeong-furiosa commented 2 years ago

I think we may need a delicate approach on this issue since, adopting greedy algorithm is improper in some situations.

ex) Assume a Renegade NPU (npu0) with 8 cores, with core indices 0, 1, 4 occupied (see diagram below).

If someone is looking for 2 singlecore devfiles, it would be best to return npu0pe2, npu0pe3.

But a step-by-step (greedy) approach may result in returning npu0pe5, npu0pe2.

image