What is the device means? VPU, HDDL, MYRIAD, HETERO, MULT

[ausk]

Hello, I've been using the openvino for a while. But I still have doubts about the device configuration passed to the demos.

• CPU, VPU, HDDL, MYRIAD, (HETERO, MULT)

The CPU means intel i5/i7, and VPU means NCS2, then

• what's the difference of HDDL and MYRIAD?
• what's the difference of HETERO and MULT?

Thank you for you time.

[ilya-lavrenov]

@ausk please, find description of the supported devices here https://docs.openvinotoolkit.org/latest/_docs_IE_DG_supported_plugins_Supported_Devices.html

[ausk]

Accoding to: https://docs.openvinotoolkit.org/latest/_docs_IE_DG_supported_plugins_VPU.html

This chapter provides information on the Inference Engine plugins that enable inference of deep learning models on the supported VPU devices:

Intel® Movidius™ Neural Compute Stick powered by the Intel® Movidius™ Myriad™ 2 
— Supported by the MYRIAD Plugin
Intel® Neural Compute Stick 2 powered by the Intel® Movidius™ Myriad™ X 
— Supported by the MYRIAD Plugin
Intel® Vision Accelerator Design with Intel® Movidius™ VPUs 
— Supported by the HDDL Plugin

According to: https://docs.openvinotoolkit.org/latest/_docs_IE_DG_supported_plugins_Supported_Devices.html

Multi-Device plugin |
    Multi-Device plugin enables simultaneous inference of the same network on several Intel® devices in parallel
Heterogeneous plugin | 
    Heterogeneous plugin enables automatic inference splitting between several Intel® devices (for example if a device doesn't support certain layers).

[ausk]

The document says:

Intel® Movidius™ Neural Compute Stick powered by the Intel® Movidius™ Myriad™ 2 
— Supported by the MYRIAD Plugin

Intel® Neural Compute Stick 2 powered by the Intel® Movidius™ Myriad™ X 
— Supported by the MYRIAD Plugin

Intel® Vision Accelerator Design with Intel® Movidius™ VPUs 
— Supported by the HDDL Plugin

---

I have a demo board with two MyriadX devices.

localhost ~ # lsusb | grep Movidius
Bus 003 Device 003: ID 03e7:2485 Movidius Ltd. Movidius MyriadX
Bus 003 Device 002: ID 03e7:2485 Movidius Ltd. Movidius MyriadX

localhost ~ # lsusb -s 003:002 -v
Bus 003 Device 002: ID 03e7:2485 Movidius Ltd. Movidius MyriadX
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass            0
  bDeviceSubClass         0
  bDeviceProtocol         0
  bMaxPacketSize0        64
  idVendor           0x03e7
  idProduct          0x2485
  bcdDevice            0.01
  iManufacturer           1 Movidius Ltd.
  iProduct                2 Movidius MyriadX
  iSerial                 3 03e72485
  bNumConfigurations      1

---

First, it's ok to run the program with MYRIAD device.

Then I test it using HDDL device. It works too. The inference times in MYRIAD and HDDL are almost the same.

But now, I can't run in MYRIAD device. If i check use lsusb, I find the device id change from 03e7:2485 to 03e7:f63b, and the bus Id also changes.

localhost ~/intel/proj_jinlj # lsusb | grep f63b
Bus 004 Device 002: ID 03e7:f63b Intel Corporation VSC Loopback Device
Bus 004 Device 003: ID 03e7:f63b Intel Corporation VSC Loopback Device

Bus 004 Device 002: ID 03e7:f63b Intel Corporation VSC Loopback Device
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               3.20
  bDeviceClass            0
  bDeviceSubClass         0
  bDeviceProtocol         0
  bMaxPacketSize0         9
  idVendor           0x03e7
  idProduct          0xf63b
  bcdDevice            1.00
  iManufacturer           1 Intel Corporation
  iProduct                2 VSC Loopback Device
  iSerial                 3 14442C103164AECC00

I just don't know what happened.

---

Then I have some questions, what the real different of HDDL and MYRIAD?

[brmarkus]

When the application starts it initializes the Myriad(X) device - it gets reset, it disappears from USB-bus (it was a high-speed USB2 device with "03e7:2485") and then it appears again on the USB-bus (now as super-speed USB3 device with "03e7:f63b"), that is expected behavior. When the application exists gracefully (or InferenceEngine/OpenVINO gets deinitialized) it will reset the Myriad(X) again, it will disappear from USB-bus and appear again.

If the application crashes than the Myriad(X) sometimes stays at "03e7:f63b". But usually it resets normally via an internal timeout. If it is a USB-stick then just unplug and re-plug it again. If it is soldered-down on the board then you might need to check the schematics to see whether the power could be turned off&on or whether there is a GPIO to access and reset the RESET pin of the device.

You can also enforce a reset of the chip via an ENV variable:

export IE_VPU_MYRIAD_FORCE_RESET=1

And then just restart your application again.

[ausk]

@brmarkus Thank you for telling the truth behind the phenomenon.

If we want to use MYRIAD after use HDDL, we can do this:

source ~/intel/openvino/bin/setupvars.sh
kill -9 $(pidof hddldaemon autoboot)
${HDDL_INSTALL_DIR}/bin/bsl_reset

---

In the description, When the application starts it initializes the Myriad(X) device - it gets reset, it disappears from USB-bus (it was a high-speed USB2 device with "03e7:2485") and then it appears again on the USB-bus (now as super-speed USB3 device with "03e7:f63b"), that is expected behavior.

Does it means, the HDDL device (usb3.2) is fast than the MYRIAD (usb2) when loading IR model?

I make an experiment to test the loading time:

Core ie;
auto network = ie.ReadNetwork(params.modelfpath);
// tic 
ExecutableNetwork executable_network = ie.LoadNetwork(network, params.devices);
// toc 

(1) Use MYRIAD:

[I] Loading model to the device ...
[T] cost: 15221.8 ms

(2) Use HDDL:

[I] Loading model to the device ...
[01:38:33.1205][6010]I[ClientManager.cpp:159] client(id:2) registered: clientName=HDDLPlugin socket=2
[01:38:48.3212][6011]I[GraphManager.cpp:491] Load graph success, graphId=2 graphName=Function_4
[T] cost: 15236.1 ms

As we can see, the time is almost the same (myriad 15221.8ms V.S. hddl 15236.1 ms) on the demo board. And when loading the IR model to the device, the cpu cost 100.0 % (1 of 4 cores, Intel E3940 @ 1.60GHz)。

The HDDL is not fast than MYRIAD when loading IR model.

[ilya-lavrenov]

@ausk do you still have questions about device names? Loading networks and time for this operation is a different story..

[ausk]

Thank you.

The device names maybe not the point, but as you mentioned, loading networks and time for this operation is a different story..

I'll close it.