search

ARM-software / ML-examples

Arm Machine Learning tutorials and examples
https://developer.arm.com/technologies/machine-learning-on-arm
Apache License 2.0
432 stars 195 forks source link

CMSIS-NN CIFAR10 example for STM32F746G-DISCO does not work? #16

Closed marcin-ch closed 5 years ago

marcin-ch commented 5 years ago

Hello All,

I am following this Image recognition on Arm Cortex-M with CMSIS-NN guide

I have exactly the same hardware:

STM32F746G-DISCO
STM32F4DIS-CAM

the same software installed

Ubuntu 16.04 LTS
Python 2.7.12
Caffe
GNU Tools for Arm Embedded Processors 7-2017-q4-major

and I am able to reproduce all mentioned in the guide steps (including building basic camera app, quantizing the model, converting the model), except the final one! = Deploy transformed model on an Arm Cortex-M processor

The final call is:

#Run this command in cmsisnn_demo folder
mbed compile -m DISCO_F746NG -t GCC_ARM --source . --source ../ML-examples/cmsisnn-cifar10/code/m7 --source ../ML-examples/cmsisnn-cifar10/camera_demo/camera_with_nn/ --source ../CMSIS_5/CMSIS/NN/Include --source ../CMSIS_5/CMSIS/NN/Source --source ../CMSIS_5/CMSIS/DSP/Include --source ../CMSIS_5/CMSIS/DSP/Source --source ../CMSIS_5/CMSIS/Core/Include -j8

The only difference is in 

--source ../ML-examples/cmsisnn-cifar10/camera_demo/camera_with_nn/

the original call refers to

--source ../ML-examples/cmsisnn-cifar10/camera_demo/camera_app/

But I have not changed camera_app folder, just used already prepared by ARM folder with camera_with_nn (ARM provides ready-to-go camera apps without NN and with NN, here)

And mbed finishes with info

[mbed] ERROR: "/usr/bin/python" returned error.

(I know, not much descriptive... but I do not see any ERRORs during building, only: multiple definition of or first defined here)

So perhaps any known issues reffering to above? Has anyone tried to complete this guide?

Hints more than welcome.

marcin-ch commented 5 years ago

Not sure if I call it a workaround but finally I have this example running. I just changed the final call to:

#Run this command in cmsisnn_demo folder
mbed compile -m DISCO_F746NG -t GCC_ARM --source . --source ../ML-examples/cmsisnn-cifar10/code/m7 --source ../ML-examples/cmsisnn-cifar10/camera_demo/camera_with_nn/ --source ../CMSIS_5/CMSIS/NN/Include --source ../CMSIS_5/CMSIS/NN/Source --source ../CMSIS_5/CMSIS/DSP/Include --source ../CMSIS_5/CMSIS/Core/Include -j8

it means I just simply cut this one:

--source ../CMSIS_5/CMSIS/DSP/Source
marcin-ch commented 5 years ago

just to close the issue and prove that I went through all the steps and got working demo :satisfied: STM32F746G-DISCO_CMSIS-NN_CIFAR10_hardware

youngolutosin commented 5 years ago

Hello marcin_ch, I am currently working on this project for my thesis. Please I need help getting it through. Do you have the link to step by step procedures so that I can follow?

marcin-ch commented 5 years ago

Hi @youngolutosin , I followed this guide Image recognition on Arm Cortex-M with CMSIS-NN guide as mentioned in my first post here. You can download this guide as PDF file (please see "Download PDF" option).

youngolutosin commented 5 years ago

I have gone through the steps and I was able to Implement the first basic camera example and it worked. I am stucked at check failed:mdb_status == 0 (2 vs.0) No file or directory. I have edited the mean_file and source file but in caffe/exmaples/cifar10 directory, I don't have mean.binaryproto, cifar10_train_lmdb and cifar10_test_lmdb. Please how did you solve the problem? Thank you in advance.

marcin-ch commented 5 years ago

Hi @youngolutosin , first of all, make sure you double-checked Troubleshooting guide

And then have a look at issue I created here with the same question as you. Hope that helps.

youngolutosin commented 5 years ago

Thank you for your response. I already figured out the problem. My project is working now. Seems the model is sensitive to some pictures. I an working on how to train the model with my own dataset and reduce the class to the specific I wanted.

Thank you and I really appreciate.

Regards, Young.

On Mon, 15 Jul 2019, 12:58 marcin-ch, notifications@github.com wrote:

Hi @youngolutosin https://github.com/youngolutosin , first of all, make sure you double-checked Troubleshooting guide https://developer.arm.com/solutions/machine-learning-on-arm/developer-material/how-to-guides/image-recognition-on-arm-cortex-m-with-cmsis-nn/troubleshooting

And then have a look at issue I created here https://github.com/ARM-software/ML-examples/issues/14 with the same question as you. Hope that helps.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ARM-software/ML-examples/issues/16?email_source=notifications&email_token=AK2TEGPX375RZYUVYQV4ABTP7RC4TA5CNFSM4HJC4N2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZ5HJNA#issuecomment-511341748, or mute the thread https://github.com/notifications/unsubscribe-auth/AK2TEGJZK4272STVYUIOTFDP7RC4TANCNFSM4HJC4N2A .

marcin-ch commented 5 years ago

@youngolutosin , yup, model is not very reliable, but they mentioned about it in the guide:

Note: that this model is very easily affected by changes in light conditions, we have chosen it only for simplicity.

And what may be also helpful for you is the keyword: transfer learning. In simple words you can use some already existing/trained model for image classification and re-train it using your dataset. It should greatly shorten the amount of time needed to train the model.

Hope that helps and good luck!

shell0108 commented 4 years ago

@marcin-ch @youngolutosin Hello ! I'm doing the same CMSIS_NN CIFAR10 example ,following the same guide ,but I met some problems.

It seems that you have done it successfully, and I will be very grateful if you offered some help! here is my _issue, 012F59C52412AB9052DD5CFD21F4F036

joshbindels commented 3 years ago

Hi @youngolutosin @marcin-ch @shell0108 ,

I'm currently working on this project for my thesis as well.

Did you notice a difference when running the model on the board or on a laptop? When I run accuracy tests using caffe or as a python script on my host machine I'm getting a decent accuracy. But, when I run a similar accuracy test on the board the accuracy is quite low, ~30%.

youngolutosin commented 3 years ago

Hi,

The accuracy of the cifar model after optimization will obviously reduce due to the conversion of the model from float32 into fixed point integer. This technique compresses the model and it affects the performance of the model.

You know the idea is to find a balance between speed, accuracy and model size so that the model will fit into the flash and ram respectively. This is very difficult to achieve.

I cannot remember the precise amount of drop in accuracy, but yes, the accuracy dropped when I ran it on the board.

You can look into STM32 CubeAI software expansion (it supports Keras, TFlowLite, Caffe, ONNX etc). Alot of improvement has been made ever since and you can achieve a better result using this.

I hope it helps.

Cheers.

On Sun, 8 Nov 2020 at 12:05, Josh Bindels notifications@github.com wrote:

Hi @youngolutosin https://github.com/youngolutosin @marcin-ch https://github.com/marcin-ch @shell0108 https://github.com/shell0108 ,

I'm currently working on this project for my thesis as well.

Did you notice a difference when running the model on the board or on a laptop? When I run accuracy tests using caffe or as a python script on my host machine I'm getting a decent accuracy. But, when I run a similar accuracy test on the board the accuracy is quite low, ~30%.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ARM-software/ML-examples/issues/16#issuecomment-723555349, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK2TEGN32GXWT223NC4JAZTSOZUNTANCNFSM4HJC4N2A .

--

Warm Regards,

Ademola Olutosin Ajibola

Master Communicative Electronics,

+37257844919,

Tallinn University of Technology, School of Information Technology,

Ehitajate tee 5, 19086, Tallinn, Estonia.

joshbindels commented 3 years ago

This looks very promising!

Thank you, I really appreciate it!

On Sun, 8 Nov 2020 at 11:17 AM, youngolutosin notifications@github.com wrote:

Hi,

The accuracy of the cifar model after optimization will obviously reduce due to the conversion of the model from float32 into fixed point integer. This technique compresses the model and it affects the performance of the model.

You know the idea is to find a balance between speed, accuracy and model size so that the model will fit into the flash and ram respectively. This is very difficult to achieve.

I cannot remember the precise amount of drop in accuracy, but yes, the accuracy dropped when I ran it on the board.

You can look into STM32 CubeAI software expansion (it supports Keras, TFlowLite, Caffe, ONNX etc). Alot of improvement has been made ever since and you can achieve a better result using this.

I hope it helps.

Cheers.

On Sun, 8 Nov 2020 at 12:05, Josh Bindels notifications@github.com wrote:

Hi @youngolutosin https://github.com/youngolutosin @marcin-ch https://github.com/marcin-ch @shell0108 https://github.com/shell0108 ,

I'm currently working on this project for my thesis as well.

Did you notice a difference when running the model on the board or on a laptop? When I run accuracy tests using caffe or as a python script on my host machine I'm getting a decent accuracy. But, when I run a similar accuracy test on the board the accuracy is quite low, ~30%.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub < https://github.com/ARM-software/ML-examples/issues/16#issuecomment-723555349 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AK2TEGN32GXWT223NC4JAZTSOZUNTANCNFSM4HJC4N2A

.

--

Warm Regards,

Ademola Olutosin Ajibola

Master Communicative Electronics,

+37257844919,

Tallinn University of Technology, School of Information Technology,

Ehitajate tee 5 https://www.google.com/maps/search/Ehitajate+tee+5?entry=gmail&source=g, 19086, Tallinn, Estonia.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ARM-software/ML-examples/issues/16#issuecomment-723556619, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABZCLFT7W7UK5VPF63UQVXLSOZV3VANCNFSM4HJC4N2A .

youngolutosin commented 3 years ago

You are welcome.

On Sun, 8 Nov 2020 at 17:02, Josh Bindels notifications@github.com wrote:

This looks very promising!

Thank you, I really appreciate it!

On Sun, 8 Nov 2020 at 11:17 AM, youngolutosin notifications@github.com wrote:

Hi,

The accuracy of the cifar model after optimization will obviously reduce due to the conversion of the model from float32 into fixed point integer. This technique compresses the model and it affects the performance of the model.

You know the idea is to find a balance between speed, accuracy and model size so that the model will fit into the flash and ram respectively. This is very difficult to achieve.

I cannot remember the precise amount of drop in accuracy, but yes, the accuracy dropped when I ran it on the board.

You can look into STM32 CubeAI software expansion (it supports Keras, TFlowLite, Caffe, ONNX etc). Alot of improvement has been made ever since and you can achieve a better result using this.

I hope it helps.

Cheers.

On Sun, 8 Nov 2020 at 12:05, Josh Bindels notifications@github.com wrote:

Hi @youngolutosin https://github.com/youngolutosin @marcin-ch https://github.com/marcin-ch @shell0108 < https://github.com/shell0108> ,

I'm currently working on this project for my thesis as well.

Did you notice a difference when running the model on the board or on a laptop? When I run accuracy tests using caffe or as a python script on my host machine I'm getting a decent accuracy. But, when I run a similar accuracy test on the board the accuracy is quite low, ~30%.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <

https://github.com/ARM-software/ML-examples/issues/16#issuecomment-723555349

, or unsubscribe <

https://github.com/notifications/unsubscribe-auth/AK2TEGN32GXWT223NC4JAZTSOZUNTANCNFSM4HJC4N2A

.

--

Warm Regards,

Ademola Olutosin Ajibola

Master Communicative Electronics,

+37257844919,

Tallinn University of Technology, School of Information Technology,

Ehitajate tee 5 <https://www.google.com/maps/search/Ehitajate+tee+5?entry=gmail&source=g , 19086, Tallinn, Estonia.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub < https://github.com/ARM-software/ML-examples/issues/16#issuecomment-723556619 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/ABZCLFT7W7UK5VPF63UQVXLSOZV3VANCNFSM4HJC4N2A

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ARM-software/ML-examples/issues/16#issuecomment-723589194, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK2TEGIYV7JELNDS64RAFT3SO2XHXANCNFSM4HJC4N2A .

--

Warm Regards,

Ademola Olutosin Ajibola

Master Communicative Electronics,

+37257844919,

Tallinn University of Technology, School of Information Technology,

Ehitajate tee 5, 19086, Tallinn, Estonia.

beyonson commented 3 years ago

Hello @marcin-ch

I know this is a very old project for you, but I am trying to figure out this camera for another project and I am having trouble with the stm32 DCMI. I was wondering if you could upload the generated bin file you get when running the lines:

cd cmsisnn_demo/ mbed compile -m DISCO_F746NG -t GCC_ARM --source . --source ../ML-examples/cmsisnn-cifar10/camera_demo/camera_app/

Although my end goal is ML, I really need to figure out how to use the camera first. I would really appreciate any advice or help, thanks!!

marcin-ch commented 3 years ago

@beyonson

where should this .bin file be located? As you mentioned, it's been a while since I touched this project last time, so I need some guidance. Is this path below right one? ~/CMSISNN_Webinar/cmsisnn_demo/BUILD/DISCO_F746NG/GCC_ARM if so, then please see attached, and please notice I have no hardware to check it on my end, so I am sending it AS IS with hope that it is working version. cmsisnn_demo.zip

NarendraKumarMadireddy commented 1 year ago

Hi @youngolutosin , I followed this guide Image recognition on Arm Cortex-M with CMSIS-NN guide as mentioned in my first post here. You can download this guide as PDF file (please see "Download PDF" option).

Can you share your project report, I am working image recognition using cortex processor with cmsis library, can you help me in completing the project, I don't have much knowledge on this topic.

marcin-ch commented 1 year ago

@NarendraKumarMadireddy seems this is the one Image recognition on Arm Cortex-M with CMSIS-NN.pdf and Youtube video as well which might be helpful Image recognition on Arm Cortex-M with CMSIS-NN in 5 steps

youngolutosin commented 1 year ago

It’s been awhile. I really do not have access to this project anymore.

On Tue, 15. Aug 2023 at 01:25, marcin-ch @.***> wrote:

@NarendraKumarMadireddy https://github.com/NarendraKumarMadireddy seems this is the one Image recognition on Arm Cortex-M with CMSIS-NN.pdf https://github.com/ARM-software/ML-examples/files/12339969/Image.recognition.on.Arm.Cortex-M.with.CMSIS-NN.pdf and Youtube video as well which might be helpful Image recognition on Arm Cortex-M with CMSIS-NN in 5 steps https://www.youtube.com/watch?v=EkYp0glSenE

— Reply to this email directly, view it on GitHub https://github.com/ARM-software/ML-examples/issues/16#issuecomment-1678164276, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK2TEGL3WTFJLAQMPKQXIQDXVKQWRANCNFSM4HJC4N2A . You are receiving this because you were mentioned.Message ID: @.***>

--

​ Regards,

Olutosin​ ​Ademola

+37253564286,

DevOps Engineer | ML Engineer

Tallinn, Estonia.

NarendraKumarMadireddy commented 1 year ago

Hello All,

I am following this Image recognition on Arm Cortex-M with CMSIS-NN guide

I have exactly the same hardware:

STM32F746G-DISCO
STM32F4DIS-CAM

the same software installed

Ubuntu 16.04 LTS
Python 2.7.12
Caffe
GNU Tools for Arm Embedded Processors 7-2017-q4-major

and I am able to reproduce all mentioned in the guide steps (including building basic camera app, quantizing the model, converting the model), except the final one! = Deploy transformed model on an Arm Cortex-M processor

The final call is:

#Run this command in cmsisnn_demo folder
mbed compile -m DISCO_F746NG -t GCC_ARM --source . --source ../ML-examples/cmsisnn-cifar10/code/m7 --source ../ML-examples/cmsisnn-cifar10/camera_demo/camera_with_nn/ --source ../CMSIS_5/CMSIS/NN/Include --source ../CMSIS_5/CMSIS/NN/Source --source ../CMSIS_5/CMSIS/DSP/Include --source ../CMSIS_5/CMSIS/DSP/Source --source ../CMSIS_5/CMSIS/Core/Include -j8

The only difference is in 

--source ../ML-examples/cmsisnn-cifar10/camera_demo/camera_with_nn/

the original call refers to

--source ../ML-examples/cmsisnn-cifar10/camera_demo/camera_app/

But I have not changed camera_app folder, just used already prepared by ARM folder with camera_with_nn (ARM provides ready-to-go camera apps without NN and with NN, here)

And mbed finishes with info

[mbed] ERROR: "/usr/bin/python" returned error.

(I know, not much descriptive... but I do not see any ERRORs during building, only: multiple definition of or first defined here)

So perhaps any known issues reffering to above? Has anyone tried to complete this guide?

Hints more than welcome.

Can you share your project report, I am working on this image recognition using cortex processor with cmsis library, can you help me in completing the project, I don't have much knowledge on this topic.

Can you share the correct detailed step by step process.

NarendraKumarMadireddy commented 1 year ago

Hello marcin_ch, I am currently working on this project for my thesis. Please I need help getting it through. Do you have the link to step by step procedures so that I can follow?

Can you share your project report, I am working image recognition using cortex processor with cmsis library, can you help me in completing the project, I don't have much knowledge on this topic.

Can you share me the step by step procedure please 🥺.

NarendraKumarMadireddy commented 1 year ago

Hi @youngolutosin @marcin-ch @shell0108 ,

I'm currently working on this project for my thesis as well.

Did you notice a difference when running the model on the board or on a laptop? When I run accuracy tests using caffe or as a python script on my host machine I'm getting a decent accuracy. But, when I run a similar accuracy test on the board the accuracy is quite low, ~30%.

Can you share your project report, I am working image recognition using cortex processor with cmsis library, can you help me in completing the project, I don't have much knowledge on this topic.

Can you share me the step by step procedure 🥺 please

youngolutosin commented 1 year ago

I don’t have access to the project anymore. Please sto spamming.

On Tue, 15. Aug 2023 at 15:31, Narendra Kumar @.***> wrote:

Hi @youngolutosin https://github.com/youngolutosin @marcin-ch https://github.com/marcin-ch @shell0108 https://github.com/shell0108 ,

I'm currently working on this project for my thesis as well.

Did you notice a difference when running the model on the board or on a laptop? When I run accuracy tests using caffe or as a python script on my host machine I'm getting a decent accuracy. But, when I run a similar accuracy test on the board the accuracy is quite low, ~30%.

Can you share your project report, I am working image recognition using cortex processor with cmsis library, can you help me in completing the project, I don't have much knowledge on this topic.

Can you share me the step by step procedure 🥺 please

— Reply to this email directly, view it on GitHub https://github.com/ARM-software/ML-examples/issues/16#issuecomment-1678853658, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK2TEGLZGHOMYGE5MLC72TDXVNT37ANCNFSM4HJC4N2A . You are receiving this because you were mentioned.Message ID: @.***>

--

​ Regards,

Olutosin​ ​Ademola

+37253564286,

DevOps Engineer | ML Engineer

Tallinn, Estonia.

NarendraKumarMadireddy commented 1 year ago

@NarendraKumarMadireddy seems this is the one Image recognition on Arm Cortex-M with CMSIS-NN.pdf and Youtube video as well which might be helpful Image recognition on Arm Cortex-M with CMSIS-NN in 5 steps

Thankyou but can you share me some report or etc , I am new to Linux and caffe

NarendraKumarMadireddy commented 1 year ago

I don’t have access to the project anymore. Please sto spamming.

On Tue, 15. Aug 2023 at 15:31, Narendra Kumar @.***> wrote:

Hi @youngolutosin https://github.com/youngolutosin @marcin-ch https://github.com/marcin-ch @shell0108 https://github.com/shell0108 ,

I'm currently working on this project for my thesis as well.

Did you notice a difference when running the model on the board or on a laptop? When I run accuracy tests using caffe or as a python script on my host machine I'm getting a decent accuracy. But, when I run a similar accuracy test on the board the accuracy is quite low, ~30%.

Can you share your project report, I am working image recognition using cortex processor with cmsis library, can you help me in completing the project, I don't have much knowledge on this topic.

Can you share me the step by step procedure 🥺 please

— Reply to this email directly, view it on GitHub https://github.com/ARM-software/ML-examples/issues/16#issuecomment-1678853658, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK2TEGLZGHOMYGE5MLC72TDXVNT37ANCNFSM4HJC4N2A . You are receiving this because you were mentioned.Message ID: @.***>

--

​ Regards,

Olutosin​ ​Ademola

+37253564286,

DevOps Engineer | ML Engineer

Tallinn, Estonia.

Sorry , thankyou

NarendraKumarMadireddy commented 1 year ago

Last question, do anyone have the windows version tutorial/PDFs/report of this project?