search

google-coral / pycoral

Python API for ML inferencing and transfer-learning on Coral devices
https://coral.ai
Apache License 2.0
347 stars 144 forks source link

Inference speed on ubuntu20.04 is more than twice as slow as inference speed on ubuntu18.04 #59

Closed arrowkato closed 2 years ago

arrowkato commented 2 years ago

Description

I use servers with AI Accelerator PCIe Card.

ubuntu18.04

I installed ubuntu18.04 (kernel version 4.18 or lower) and configured it according to Get started with the M.2 or Mini PCIe Accelerator

At this time, I installed libedgetpu1, not libedgetpu1-std, but libedgetpu1-max. In particular, the following command.

sudo apt-get install gasket-dkms libedgetpu1-max

After setting this up, I ran the inference process multiple times, and it took about 4 milliseconds after the second inference. It varies depending on the inference process, but it is between 3.5 and 4.5 milliseconds.

ubuntu20.04

On the same physical server, I reinstall ubuntu20.04 (kernel version 4.19 or higher) and run in this manual I set it up in the same way, including the car section.

I reinstalled ubuntu20.04 (kernel version 4.19 or higher) on the same physical server and set it up, including "Workaround to disable Apex and Gasket" section of Get started with the M.2 or Mini PCIe Accelerator I installed libedgetpu1-max as I did on ubuntu18.04

sudo apt-get install gasket-dkms libedgetpu1-max

In this case, when I let the inference process run multiple times, it took about 11.2 milliseconds after the second inference. It varies depending on the inference process, but it is between 11.0 and 11.4 milliseconds.

Requests

Is there a setting to make ubuntu20.04 infer at the same speed as ubuntu18.04 (about 4ms)? If there are problems on the pycoral side, I want them fixed.

Click to expand! ### Issue Type Performance ### Operating System Ubuntu ### Coral Device _No response_ ### Other Devices _No response_ ### Programming Language Python 3.6, Python 3.9 ### Relevant Log Output _No response_
manoj7410 commented 2 years ago

@arrowkato Please paste the output of below commands from both the machines:

arrowkato commented 2 years ago

The configuration with Ubuntu 20.04 and Python 3.9.7 and pycoral 2.0.0 was slow to the extent described top message.

The configuration of Ubuntu20.04 and Python3.9.10 and pycoral 2.0.0 was almost as fast as the configuration of ubuntu18.04 and python3.6 and pycoral1.0.1.

So the problem for us was solved.

Thank you.

google-coral-bot[bot] commented 2 years ago

Are you satisfied with the resolution of your issue? Yes No