Using Mobilenet

[MaRcOsss1]

Hey Andre, I had a doubt regarding Model Conversion. I am trying to use Mobile net trained model, which is already based on Tensorflow, Do I need to undergo model conversion for that? I dont think I should , under that case what could be my next step? Should I move to quantization?

So do you know where I can refer to scripts for quantization? or atleast I can get Idea how to quantize my mobilenet model?

[andre1araujo]

Hi! If your model is already in tensorflow, you can move to the quantization process directly. I assume you have MobileNet model in a ".pb" format, so, you should move that file to the quantization folder. Important note: DNNDKv3.1 is the quantization and compilation tool of Xilinx and that version supports Tensoflow 1.12! The model has to respect that version otherwise it will not work. There might be a way to downgrade the model if it is in Tensorflow 2 as most of the implementations. This is maybe the biggest limitation of the deployment of neural networks on the PYNQ-Z2 with this tool. If you will be using MobileNet for image classification and not object detection, you have most of the quantizatization process like I describe on the tutorial. Follow everything correctly but you will need to change the calibration script. I don't think that task is too hard so you can try to use chatGPT for that. On the other hand there might be some work done on that regard before. Just try to look up for implementations of MobileNet or other similar classification models but implemented on simillar hardware, like the Ultra96 board. Also, the DNNDK packages you download has some examples that might be useful! I hope this helps. Fell free to ask any questions if you find further problems! :D

[MaRcOsss1]

Hey Andre, Okay I will check that process myself. Also, if I plan to implement object detection using mobilenet what do I have to do differently? What could possibly change?

[andre1araujo]

If you are talking about MobileNet, it is a classification model. It just tells you what is on the image. There is SSD MobileNet which is a model for object detection which has a base on MobileNet. SSD will later require a script in c++ that might be considerably different from the one I have for YOLO. In any case, there are certainly pre-made example scripts from other people or from Xilinx. When I have time I can looks into that. Good luck!

[MaRcOsss1]

Hey Andre, I was setting up the DNNDK and was installing the necessary packages : pip install numpy opencv-python==4.7.0.68 scikit-learn==0.24.2 scipy progressbar2 keras==2.2.4 and this happened :

Whats the problem here?

Also I was trying to convert the darknet model to keras using this command python convert.py yolov3.cfg yolov3.weights /home/<user>/Desktop/YOLO-on-PYNQ-Z2/DNNDK/model_conversion/model_data/yolo.h5 and ran into this error :

aryan@aryan-X510UAR:~$ python convert.py yolov3.cfg yolov3.weights /home/aryan/Desktop/YOLO-on-PYNQ-Z2/DNNDK/model_conversion/model_data/yolo.h5

Command 'python' not found, but can be installed with:

sudo apt install python3
sudo apt install python
sudo apt install python-minimal

You also have python3 installed, you can run 'python3' instead.

So I installed python and tried to re-run and this is the error:

aryan@aryan-X510UAR:~$ python convert.py yolov3.cfg yolov3.weights /home/aryan/Desktop/YOLO-on-PYNQ-Z2/DNNDK/model_conversion/model_data/yolo.h5 python: can't open file 'convert.py': [Errno 2] No such file or directory

But the convert.py file is already in the given path:

[andre1araujo]

Hi! I have PTSD of that first error because I spent more than two weeks trying to deal with it haha. But whan that happens is due to incompatibilities of the packages. In this case, I went and checked the opencv-python package and aparently it deprecated:

What you need to do now is change the version to what they suggest there 4.7.0.71. pip install numpy opencv-python==4.7.0.71 scikit-learn==0.24.2 scipy progressbar2 keras==2.2.4 If it does not work, experiment with the version before that, which is 4.7.0.66. If that does not work too, you will have to try and downgrade or upgrade the rest of the packages. Give it a try and then update me if it worked so I can change the tutorial. Thank you!

The second problem is because you are not running the script convert.py on the same folder as the picture. You first have to access the directory of the python scrip like this: cd /home/aryan/Desktop/YOLO-on-PYNQ-Z2/DNNDK/model_conversion And then you do the conversion: python convert.py yolov3.cfg yolov3.weights /home/aryan/Desktop/YOLO-on-PYNQ-Z2/DNNDK/model_conversion/model_data/yolo.h5

I hope this helps. Good luck!

[MaRcOsss1]

so apparently the version 4.6.0.66 is working for now , I tired versions from 4.7 to 4.10 none of them worked but this one is wokring as you can see :

aryan@aryan-X510UAR:~/Downloads/xilinx_dnndk_v3.1/host_x86/decent-tf/ubuntu18.04$ pip install numpy opencv-python==4.6.0.66 scikit-learn==0.24.2 scipy progressbar2 keras==2.2.4 Requirement already satisfied: numpy in /home/aryan/anaconda3/envs/decent/lib/python3.6/site-packages (1.19.5) Collecting opencv-python==4.6.0.66 Downloading opencv_python-4.6.0.66-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (60.9 MB) |████████████████████████████████| 60.9 MB 54 kB/s Collecting scikit-learn==0.24.2 Using cached scikit_learn-0.24.2-cp36-cp36m-manylinux2010_x86_64.whl (22.2 MB) Collecting scipy Using cached scipy-1.5.4-cp36-cp36m-manylinux1_x86_64.whl (25.9 MB) Collecting progressbar2 Using cached progressbar2-3.55.0-py2.py3-none-any.whl (26 kB) Collecting keras==2.2.4 Using cached Keras-2.2.4-py2.py3-none-any.whl (312 kB) Collecting joblib>=0.11 Using cached joblib-1.1.1-py2.py3-none-any.whl (309 kB) Collecting threadpoolctl>=2.0.0 Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB) Requirement already satisfied: six>=1.9.0 in /home/aryan/anaconda3/envs/decent/lib/python3.6/site-packages (from keras==2.2.4) (1.16.0) Requirement already satisfied: keras-applications>=1.0.6 in /home/aryan/anaconda3/envs/decent/lib/python3.6/site-packages (from keras==2.2.4) (1.0.8) Requirement already satisfied: h5py in /home/aryan/anaconda3/envs/decent/lib/python3.6/site-packages (from keras==2.2.4) (3.1.0) Collecting pyyaml Using cached PyYAML-6.0.1-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (677 kB) Requirement already satisfied: keras-preprocessing>=1.0.5 in /home/aryan/anaconda3/envs/decent/lib/python3.6/site-packages (from keras==2.2.4) (1.1.2) Collecting python-utils>=2.3.0 Using cached python_utils-3.5.2-py2.py3-none-any.whl (24 kB) Requirement already satisfied: typing-extensions in /home/aryan/anaconda3/envs/decent/lib/python3.6/site-packages (from python-utils>=2.3.0->progressbar2) (4.1.1) Requirement already satisfied: cached-property in /home/aryan/anaconda3/envs/decent/lib/python3.6/site-packages (from h5py->keras==2.2.4) (1.5.2) Installing collected packages: threadpoolctl, scipy, pyyaml, python-utils, joblib, scikit-learn, progressbar2, opencv-python, keras Successfully installed joblib-1.1.1 keras-2.2.4 opencv-python-4.6.0.66 progressbar2-3.55.0 python-utils-3.5.2 pyyaml-6.0.1 scikit-learn-0.24.2 scipy-1.5.4 threadpoolctl-3.1.0

[andre1araujo]

Alright, thank you! I will update the tutorial with that information. I hope everything will work accordingly for now on! Good luck!

[MaRcOsss1]

Why did this happen?

Later I tried to run : sudo chmod +x quant.sh and it showed quantization finished in less than 10-15 seconds????? It is showing some Name error you can see at the end of the log: Maybe it is regarding the output nodes :

So I changed the output nodes in the quant.sh file I hope it is all right?

[MaRcOsss1]

Hey, It showed Quantization ended only after 3/100

A folder anmed Quantization result is made but is empty:

And a pycache named folder is created which included a .pyc file:

[andre1araujo]

Hello Aryan. Sorry for the late response. I think you should first do this command to see the input nodes, output nodes and shape: decent_q inspect --input_frozen_graph=model_data/yolo.pb. I mention this on the tutorial and it is important, especially because you are using a different model. Complete the quant.sh script with the informations displayed by the command. The COCO dataset shpuld be uncompressed. It takes some time, yes haha. The images are the 17GB I think. Try to see if the names of the inputs and outputs of the network with the command and complete the script with the information. Then let me know everyting! I am sorry if I take some time to answer but for now I tend to be a little more busy. I will do my best! :) Good luck!

[MaRcOsss1]

So I had ran that and I had updated the output nodes accordingly in the quant.sh, input nodes , output nodes everything is fine and right (here). The problem is the quantization isn't getting completed, it stops at 1,2 or 3 (out of 100). Is it possible that my Computer isn't capable of doing that? (here) Or Should I try another dataset with less images? Can you recommend any?

It's okay take your time, I can understand you can be busy and I appreciate you for still helping out.

[andre1araujo]

It might be because of computational power. I am not sure but still, try to reduce the "calib_iter" to a lower number like 50 or even 10. Reducing the batch size generaly helps. You are also using the cpu so it is expected to be more demanding. I hope this helps!

[MaRcOsss1]

I don't think reducing the caIib_iter would make a difference, it is because the size of dataset mainly, so maybe reducing batch size might help?

And I have access to a better system but it runs on Ubuntu v20.04 , is it okay? or should I stick with the v18.04 itself?

[andre1araujo]

Yes, reducing the size of the batch helps reduce computational load. Normally computers with more computational power can use bigger batches, specialy with a GPU. Unfortunatly, for DNNDK you can only work with Ubuntu 14.04, 16.04 or 18.04. Not Ubuntu 20.04. It says on the documentation here (page 8):

Good luck!

[MaRcOsss1]

F okay. About the Batch size I reduced it to 10 instead of 50 and calib_iter as 20 and It worked, does it mean it takes 10 photos and then another 10 and so on, till 20, am I understanding this right or am I making a mistake?

Also I got on the step on compilation just to know the process but I ran into error as always :

[andre1araujo]

Hi! You are understanding it right. What you said is correct. About the issue on compilation, it is a library that you need to install but I forgot to include it on the tutorial. Install it like this: sudo apt install libgoogle-glog-dev I will now update the tutorial for that. I am sorry for all the problems you are encountering. Good job!

[MaRcOsss1]

Hardly any problems I encountered are on you, all of them are mostly on me and to be honest upon our discussions from past 2-3 weeks all of them has been solved so don't take it on yourself. When I encountered this error today , First thing that came to my mind was what the heck did I do wrong again haha. Thank you so much Andre, you're the best.

On Wed, Jul 10, 2024, 8:37 PM André Araújo @.***> wrote:

Hi! You are understanding it right. What you said is correct. About the issue on compilation, it is a library that you need to install but I forgot to include it on the tutorial. Install it like this: sudo apt install libgoogle-glog-dev I will now update the tutorial for that. I am sorry for all the problems you are encountering. Good job!

— Reply to this email directly, view it on GitHub https://github.com/andre1araujo/YOLO-on-PYNQ-Z2/issues/7#issuecomment-2220762311, or unsubscribe https://github.com/notifications/unsubscribe-auth/A5RZEIXUDYKQSRMJK35ZX3LZLVE2BAVCNFSM6AAAAABKG42TVWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRQG43DEMZRGE . You are receiving this because you authored the thread.Message ID: @.***>

[andre1araujo]

No problem haha. This error in particular, somehow I still remember it and I quickly checked it on my notes and there it was. Keep pushing and any problem just tell me because you are helping a lot here.

[MaRcOsss1]

Hey, is this warning serious? Or I should ignore it?

Okay I checked turns out it is , I had to run sudo apt-get install graphviz this to get the .jgp file, you can add this as well on your tutorial.

Deployment part onwards everything is on windows, right? So should I copy the files generated in the Compilation step from Ubuntu to windows?

[andre1araujo]

The warning you got is not very important. After all you just get a image with the DPU kernel. I don't know exactly how it is useful but maybe in some applications it might be. In any case, I will add that command. Thank you! The rest of the process is on Windows indeed. Copy the files from the compilation to the Windows system and do the rest there. I hope there are not much problems now! Good luck!

[MaRcOsss1]

without that library you won't get a .jpg file. I checked after the warning , and mine wasn't converted to .jpg file. After downloading that library, .jpg was generated.

[andre1araujo]

Okay, thank you! Already updated the tutorials!