Wow

[StuartIanNaylor]

Trying to install armnn on Raspios64 which is a mine field of broken documentation that errors out after epic compile waits. Ubuntu 22.04 OK new Switched to 20.04 but surely you would have debs and a ppa for RaspiOS 64 Bullseye as that has been some time now. The repo version is so old its pre tflite delegate.

Never installed anything by Arm before and 1st impressions is your documentation and repo updates are shocking.

Anyway back to 20.04 as will persevere but honestly and this isn't rage its been a nightmare and it is the 1st time I have used the software hopefully the speedup boosts over tflite is true after all this :)

Stuart

[morgolock]

Hi @StuartIanNaylor

Sorry to hear you are having problems with the documentation. Could you please share with us some of the issues you found so that we can fix the problems?

[StuartIanNaylor]

Well start with https://arm-software.github.io/armnn/22.02/ as near all of it from debian/ubuntu repos to compiles that break with errors is all out of date and the simple gettiing started of copy the release to a folder and run a delegate is actually missing. But when you do eventually find where the delegate method is mentioned as per usual the tutorial can not be copied and pasted as it has the wrong version number which is annoying as wy not use the non versioned sym link? In every and I mean every tutorial you do on armnn there is not one that doesn't miss something or correct in entirety but the biggest problem is you don't even document the preferred delegate method and never prune out old documentation and much what you have is actually defunct or at least not that relevant.

Get an Arm fresher with little experience to record there experiences as I have no wish to go through those very long pi4 compiles once more as really its not really necessary. https://developer.arm.com/documentation/102561/2111/Overview/Arm-NN-TensorFlow-Lite-Delegate?lang=en This should be the documentation and someone should actually test if it works and maybe actually name and place somewhere easily acessable.

"Image Classification with MobilenetV2, Arm NN, and TensorFlow Lite Delegate pre-built binaries Tutorial` it took so long to find that but that is the easiest way and supposedly your preferred way but is totally absent from armnn

[StuartIanNaylor]

PS after my experience of Arm documentation I thought I would contact

Dear Stuart,

For me, it was the same nightmare to get ARMnn up and running. No one has commented on the installation ever since. You're the first. Nor uses the GitHub example. I didn't pay any attention to it. At the time of writing, there were other issues as well. Not all the TF models give the same outcomes. Moreover, you need TF installed before even getting ARMnn working.

With regards,

Rients.

Hopefully he will not mind me sharing but https://qengineering.eu/ are a amazing resource for all sorts of install tutorials for AI and prob a reason why he has had zero interest for ArmNN is your documentation is so badly presented and disjointed prob apart from handheld paying customers is using it.

[james-conroy-arm]

Hi Stuart,

Thanks for your feedback.

Am I correct in saying that you'd prefer more (relevant) build guides on our GitHub repo, instead of on the Arm Developer website?

Cheers, James

[StuartIanNaylor]

Both prominent as really your compile guides have been replaced for preference for delegate and are obsolete.

Still haven't checked if the speed up claims are correct yet but in terms of the next gen of AI developers your current documentation is pushing them away as just get the binaries and ...

# Load TFLite model and allocate tensors
armnn_delegate = tflite.load_delegate(library="/usr/lib/libarmnnDelegate.so",
options={"backends": "VsiNpu, CpuAcc, CpuRef",
"logging-severity": "info"})
# Delegates/Executes all operations supported by ArmNN to/with ArmNN
interpreter = tflite.Interpreter(model_path="mobilenet_v1_1.0_224_quant.tflite",
experimental_delegates=[armnn_delegate])

That is all that is needed but for some reason have a plethora of documents of compile hell :)

A working build guide would be good but no actually make it known there is a really quick and easy way 1st. Also with https://developer.arm.com/documentation/102561/2111/Overview/Arm-NN-TensorFlow-Lite-Delegate?lang=en maybe not have an incorrect file name for the delegate and when doing a demonstration on a tflite model and actually check it works and is tflite as doesn't and isn't.

[james-conroy-arm]

Hi Stuart,

Thank you for using our software with the 64-bit Raspberry Pi OS - this platform is a key focus for us.

To date, we have been working hard to make it easier for developers to use Arm NN through the use of our pre-built binaries and our Arm NN TF Lite Delegate. Whilst this is now in place, we haven't made this clear enough in our documentation. We will address this as soon as possible, for example by making our GitHub homepage similar to what is shown on ACL's page: https://github.com/ARM-software/ComputeLibrary#pre-built-binaries

We also have plans in our roadmap to open source a scripted solution to build Arm NN from scratch. This means that our current build guides will be replaced with automated parameterized scripts. We hope this will ease the pain you have described when building Arm NN from scratch.

We are interested in knowing how you progress with the pre-built binaries and would love to get feedback on the performance speedup you experience.

Thanks again for your feedback.

Cheers, James

[StuartIanNaylor]

No probs as that is all I want to convey as if the likes of me and especially https://qengineering.eu/ struggled most will just switch off also https://github.com/PINTO0309 as prob you forwarded out info to them it would fan out quickly.

I am really interested about using the Mali and various NPUs on non raspbian 'embedded' SBC no just CPU & Pi and that should be easier and more transparent. I have a G31 MP2 / G52 MP4 & hopefully when I get hold of a RK3588 one of your newer G610 and really interested on how the delegate can work and think others would be if it wasn't such a headache when really it isn't to implement.

Best wishes Stuart

[StuartIanNaylor]

@james-conroy-arm PS on another tangential thought https://github.com/ARM-software/ML-KWS-for-MCU kick started much activity and Google replied with https://github.com/google-research/google-research/tree/master/kws_streaming which includes much more recent models and methods from spectral augmentation to streaming.

The original https://github.com/ARM-software/ML-KWS-for-MCU and model zoo members are now starting to look elderly and would be great if not just for MCU we had a refresh of ARM-software/ML-KWS and not such focus on MCU

[StuartIanNaylor]

@james-conroy-arm ps on ubuntu 20.04 the tutorial https://developer.arm.com/documentation/102561/2111/Overview/Arm-NN-TensorFlow-Lite-Delegate?lang=en works

On RaspiOS Bullseye the latest release and tutorial release load but end with a seg fault.

Info: ArmNN v27.0.0

Can't load libOpenCL.so: libOpenCL.so: cannot open shared object file: No such file or directory
Can't load libGLES_mali.so: libGLES_mali.so: cannot open shared object file: No such file or directory
Can't load libmali.so: libmali.so: cannot open shared object file: No such file or directory
Couldn't find any OpenCL library.
Info: Initialization time: 1.73 ms

INFO: TfLiteArmnnDelegate: Requested unknown backend  CpuRef
INFO: TfLiteArmnnDelegate: Created TfLite ArmNN delegate.
Segmentation fault

[james-conroy-arm]

Hi @StuartIanNaylor

We expedited some of our planned changes to our documentation which were in our roadmap and which also reflect your feedback. The changes are now live on our GitHub homepage and will be included in the 22.05 Arm NN Release at the end of this month: https://github.com/ARM-software/armnn/tree/master (this is a link to the master branch of Arm NN)

Regarding KWS - I will raise this with the appropriate team within Arm.

Regarding TF Lite Delegate, we will address this in your other opened ticket: https://github.com/ARM-software/armnn/issues/647

Thanks again for your feedback, it is important to us.

Cheers, James

[StuartIanNaylor]

The delegate method of automatically by-passing non supported layers & methods is just really great for Arm developers as then its just instant bolt-on acceleration for layers that it does. Problem is now its not just the errors and faults have declined access for testing as they have severely dented my confidence that all is well with Arm with this product.

I have actually been wondering if the articles where purely a carrot for paywall software services than anything concrete.

[StuartIanNaylor]

It just seems to me they are purely tempters for Arm Developer services and you don't do 100% working guides anywhere...

[keidav01]

Hi @StuartIanNaylor,

Sincerest apologies for this. I have just come across your issue myself, however as it happens, we are currently reviewing our documentation for users like yourself to get up and running properly. This is valuable information and I will pass onto the relevant parties.

Thank you for the information and we will do our best to fix this as soon as possible

[james-conroy-arm]

Hi @StuartIanNaylor ,

I would echo @keidav01 's comment above - we are working hard to improve our documentation and always welcome feedback.

Since my last comment on this ticket, we released the Arm NN Build Tool in response to developers' issues with our build documentation. We have an open issue where developers can leave feedback on this tool, please feel free: https://github.com/ARM-software/armnn/issues/689

Please let us know if you require assistance with anything else.

Cheers, James

[StuartIanNaylor]

Will do as got a bit frustrated :) apols.