eembc / mlmark

EEMBC's Machine-Learning Inference Benchmark targeted at edge devices.
https://www.eembc.org/mlmark
Other
45 stars 5 forks source link

running models on Myriad target #6

Open kselvan079 opened 4 years ago

kselvan079 commented 4 years ago

Hey, i have been trying to run the models on different targets, and it has been successful in cpu targets. But when i try to run on myriad using openvino target i get an error. The throughput files have completed successfully, but the accuracy file stops specifically on above specified target with the following error Screenshot from 2019-09-11 15-32-49

petertorelli commented 4 years ago

What platform are you using, NCS or NCS2? I have seen cases where the NCS2 randomly fails if not connected to a USB3.0 port. Are you using a 3.0 port?

kselvan079 commented 4 years ago

Its NCS2 and i made sure its connected to the USB 3.0 port. I have tried multiple times and failed at different points. At first i thought image was the problem, but that possibility was ruled out when i was able to run same images on cpu target. Any other possibilities?

rkalidin commented 4 years ago

Can you please provide Host system configuration and error log (Last line seems to be cut off in the screen shot)

kselvan079 commented 4 years ago

H/W path Device Class Description

                        system         80TL (LENOVO_MT_80TL_BU_idea_FM_V110-

/0 bus LNVNB161216 /0/2 memory 64KiB L1 cache /0/3 memory 64KiB L1 cache /0/4 memory 512KiB L2 cache /0/5 memory 3MiB L3 cache /0/6 processor Intel(R) Core(TM) i3-6006U CPU @ 2.00 /0/7 memory 8GiB System Memory /0/7/0 memory 4GiB SODIMM DDR4 Synchronous 2133 MHz /0/7/1 memory [empty] /0/7/2 memory 4GiB SODIMM DDR4 Synchronous 2133 MHz /0/7/3 memory [empty] /0/d memory 128KiB BIOS /0/100 bridge Skylake Host Bridge/DRAM Registers /0/100/2 display HD Graphics 520 /0/100/14 bus Sunrise Point-LP USB 3.0 xHCI Control /0/100/14/0 usb1 bus xHCI Host Controller /0/100/14/0/5 multimedia EasyCamera /0/100/14/0/8 communication Bluetooth wireless interface /0/100/14/0/9 generic USB2.0-CRW /0/100/14/1 usb2 bus xHCI Host Controller /0/100/14.2 generic Sunrise Point-LP Thermal subsystem /0/100/16 communication Sunrise Point-LP CSME HECI #1 /0/100/17 storage Sunrise Point-LP SATA Controller [AHC /0/100/1c bridge Intel Corporation /0/100/1c.4 bridge Sunrise Point-LP PCI Express Root Por /0/100/1c.4/0 wlp2s0 network Dual Band Wireless-AC 3165 Plus Bluet /0/100/1d bridge Intel Corporation /0/100/1d/0 enp3s0 network RTL8111/8168/8411 PCI Express Gigabit /0/100/1f bridge Sunrise Point-LP LPC Controller /0/100/1f.2 memory Memory controller /0/100/1f.3 multimedia Sunrise Point-LP HD Audio /0/100/1f.4 bus Sunrise Point-LP SMBus /0/0 scsi0 storage
/0/0/0.0.0 /dev/sda disk 1TB ST1000LM035-1RK1 /0/0/0.0.0/1 /dev/sda1 volume 928GiB EXT4 volume /0/0/0.0.0/2 /dev/sda2 volume 3519MiB Extended partition /0/0/0.0.0/2/5 /dev/sda5 volume 3519MiB Linux swap volume /0/1 scsi1 storage
/0/1/0.0.0 /dev/cdrom disk DVDRAM GUE0N /1 power Smart Battery /2 docker0 network Ethernet interface Screenshot from 2019-09-17 15-47-11

i have pasted the screenshot and also the text snippet for your convenience. PS: i suspect that my usb port 3.0 is faulty, will update you on that, after trying on another host system.

kselvan079 commented 4 years ago

So i have ran the usb 3.0 diagnostics and and tested the same model on another host as well and got the same results. Seems that its not a hardware problem. So i ran the model on i3 6th gen processor, 8 gb ram with usb 3.0 port on ncs2. I have two issues.

1) The throughput of mine is half of the score mentioned in the embassy for all models on ncs2. 2) the accuracy.json file takes 3 hrs and 40 mins to complete. Is it the expected?

petertorelli commented 4 years ago

The very long accuracy runtime is a known issue. What version openVino are you using ('/opt/intel/...')? As for half the performance, I do not know. @rkalidin any ideas?

kselvan079 commented 4 years ago

I am using openvino_2019.1.144. Even after running on multiple hosts, the result is same. The throughput is half of the results posted in embassy page.