search

ARM-software / armnn

Arm NN ML Software. The code here is a read-only mirror of https://review.mlplatform.org/admin/repos/ml/armnn
https://developer.arm.com/products/processors/machine-learning/arm-nn
MIT License
1.14k stars 307 forks source link

Batch Image Inference using PyArmNN Error #655

Closed AliSheheryar closed 1 year ago

AliSheheryar commented 2 years ago

I am trying to have a batch inference on my image list( 2 images) using Pyarmnn on Arm-v7l device. Single Image inference is doing fine. make_input_tensor function produces the required tensors but when fed to enqueue_workload function, it throws a runtime error.

File "input_tensor_testing.py", line 172, in main runtime.EnqueueWorkload(net_id, input_tensors, output_tensors) File "/usr/lib/python3/dist-packages/pyarmnn/_generated/pyarmnn.py", line 4118, in EnqueueWorkload return _pyarmnn.IRuntime_EnqueueWorkload(self, networkId, inputTensors, outputTensors) RuntimeError: Number of inputs provided does not match network.

I am using Tflite file for Yolo_v3 for object detection.

MikeJKelly commented 2 years ago

Hi @AliSheheryar

how are you passing the two images? Are you passing them one image at a time or both at once? If you want to perform inferences on two images you will have to call enqueue_workload twice, once with each image.

Best regards, Mike

AliSheheryar commented 2 years ago

Mike thanks for your help. I want to run inference on two images at the same time. enqueue_workload function can also work on list on input images a/c to ARMNN package documentation.A little explanantion would be helpful. Furthermore, on CPUAcc as backend , what is the maximum FPS you can achieve using CPU only? I am getting 1FPS and wondering if there is another way to enhance FPS. GPU is not an option as ARMv7 devices don't have NVIDIA GPU power.

FrancisMurtagh-arm commented 1 year ago

Hi @AliSheheryar,

We would recommend upgrading to an ARMv8 device as most of our optimizations such as those related to NEON are not supported on ARMv7.

Could you provide a code snippet of what you are trying to do re: "to run inference on two images at the same time".

Thanks, Francis.

AliSheheryar commented 1 year ago

Thank you for you cooperation. I was able to run the armv7 with 1Frame/dev speed with Neon acceleration module but no GpU. I can provide the whole code and stuff. Open source is the way to move forward.

On Fri, Jan 6, 2023 at 7:02 PM FrancisMurtagh-arm @.***> wrote:

Hi @AliSheheryar https://github.com/AliSheheryar,

We would recommend upgrading to an ARMv8 device as most of our optimizations such as those related to NEON are not supported on ARMv7.

Could you provide a code snippet of what you are trying to do re: "to run inference on two images at the same time".

Thanks, Francis.

— Reply to this email directly, view it on GitHub https://github.com/ARM-software/armnn/issues/655#issuecomment-1373685188, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANO5JNONF3KUULDFZPBICE3WRAQW5ANCNFSM5ZIFJNSQ . You are receiving this because you were mentioned.Message ID: @.***>

AliSheheryar commented 1 year ago

Not a batch of images. Just one image per function call. Sorry for not answering earlier about the actual question.

On Sat, Jan 7, 2023 at 6:04 PM Ali Sheheryar @.***> wrote:

Thank you for you cooperation. I was able to run the armv7 with 1Frame/dev speed with Neon acceleration module but no GpU. I can provide the whole code and stuff. Open source is the way to move forward.

On Fri, Jan 6, 2023 at 7:02 PM FrancisMurtagh-arm < @.***> wrote:

Hi @AliSheheryar https://github.com/AliSheheryar,

We would recommend upgrading to an ARMv8 device as most of our optimizations such as those related to NEON are not supported on ARMv7.

Could you provide a code snippet of what you are trying to do re: "to run inference on two images at the same time".

Thanks, Francis.

— Reply to this email directly, view it on GitHub https://github.com/ARM-software/armnn/issues/655#issuecomment-1373685188, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANO5JNONF3KUULDFZPBICE3WRAQW5ANCNFSM5ZIFJNSQ . You are receiving this because you were mentioned.Message ID: @.***>

matthewsloyanARM commented 1 year ago

Hi @AliSheheryar,

Thanks for getting in touch. If this is still an issue, would you be able to provide a code snippet here and I can try and reproduce it for you?

Kind regards,

Matthew

matthewsloyanARM commented 1 year ago

Hi @AliSheheryar,

I am going to close this issue as I haven't heard back in a little while, however if you are still having the issues feel free to reopen the ticket. Thanks again for getting in touch.

Kind regards,

Matthew