camera

[mark2146]

Hi, thank you very much for your help with the project. I’ve successfully used it, but I have two questions I'd like to ask. First, when using the camera, I’m experiencing disconnection issues, as shown in the image, and the image feed is very slow. Is this an issue with my camera, or is there something I can adjust because it seems our FPS are different? Second, is it possible to replace the YOLO model in the project with one that I have trained myself? Thank you

[andre1araujo]

Hello! I am sorry for the delayed response.

The problem related with the board suddenly shutting off is due to power issues. I recommended on the tutorial the use of a external power source (a normal 12V transformer) because the application is very demanding and the USB power is more inconsistent. That should solve the problem completely.

About the latency of the application, it is expected to be very slow for a real time cenario. The DPU is responsible for processing the YOLO neural network which is very complex and has a lot of computations but the resources of the board don't allow for something faster, at least for now. The latency does not have anything to do with your camera! I should mention that the FPS value might be different because I did not had time to change FPS count on the c++ program. I had a little error and the timer start should be placed somewhere earlier on the code. If you want a faster application, I recommend you try to use Yolov2 which is a low less complex and should result on a better real time performance.

You can absolutely use your own model for this! If you trained your YOLOv3 model, you just have to do this part to convert it to keras, then to Tensorflow and follow the rest of the process. If your model has less classes it can actually be a little faster than the current version!

I hope this helps you! If you have any more questions feel free to ask on this thread or on a new issue! :)

[mark2146]

If I understand correctly, you have successfully run the DPU and DNNK on the PYNQ Z2. Now you are wondering if you can use your TensorFlow model directly instead of running the YOLO neural network model. If so, is there a tutorial available? I have searched online and found that most resources are about CNN models, which are a bit too complex for me.Thank you

[andre1araujo]

Hello! I am not sure if I understand your question correctly. Some context might have been lost on the translation. The Tensorflow model is the yolo.pb file. This file contains the YOLO neural network inside. Any ".pb" file (Tensorflow file) that you use can be converted to a ".elf" file and executed on the PYNQ-Z2. For example, you can use the model "VGG19" for image classification on the PYNQ-Z2. What you will have to do is find a VGG19 model online with the ".pb" extension and then replicate the tutorial. Two things to take in consideration are the ".pb" file version (it has to be 1.12 or less) and the calibration dataset (you should use for example Imagenet because is a dataset for image classification). I know it can seem a bit difficult for someone with no experience but I a, here to help! :) Any questions you have, jut feel free to ask!

[mark2146]

My idea is to use Edge Impulse to train a model for use on the PYNQ-Z2. However, since the PYNQ-Z2 does not support Edge Impulse directly, I have been struggling to find a solution. I noticed that the models trained with Edge Impulse are in TensorFlow format. Is it possible to apply these models directly to the PYNQ-Z2?（Edge Impulse is a tool for training TinyML models.）

[andre1araujo]

Okay, so I looked up about the Edge Impulse services and specifically looked to this Tutorial for the Kria-som board. It seems that the application makes the training process for custom datasets really easy. That is all really good but unfortunatly I see some limitations that will make it impossible to do it on the PYNQ-Z2. The problem is that it works with Tensorflow but I suspect is a recent version. Xilinx DNNDK stopped supporting the Zynq7000 series chips (the one on the PYNQ-Z2) some years ago and so my implementation is limited to the older Tensorflow 1.12. As Edge impulse outputs a Tensorflow model on a more recent version, it cannot be done with DNNDK. The applications to embed CNN models on the Zynq7000 familly chips is quite limited nowadays as the industry is shifting to some more powerfull FPGAs with more resources.

[mark2146]

So according to this theory, if I use TensorFlow 1.12 to train a model myself, will it be possible to use it on the PYNQ-Z2? I am currently working on a project and can't find many hardware-related resources online. Thank you for your help!

[andre1araujo]

Absolutely! If you train a model in Tensorflow 1.12 or lower, you should be able to follow the tutorial and use it on the PYNQ-Z2 (accelerated on the FPGA). If you are able to get the model I can help you with the proceeding steps. Good luck!

[mark2146]

Thank you for your help. I will first use TensorFlow version 1.12 to train a small model. I’ll ask for your guidance when the time comes. I really appreciate it!

[mark2146]

I'm sorry to ask such basic questions, but how did you successfully install TensorFlow 1.12? I downgraded my Python to 3.6, but I still cannot execute it successfully. It always reports that it's not supported. I also couldn't get the .whl file installation to work, and I haven't found a correct solution online. Could you help me with this?

[andre1araujo]

Hi! I did not install Tensorflow 1.12 directly. We work with DNNDK tool and it uses Tensorflow 1.12 to perform some of the operations. I should note that you have to use Ubuntu 16.04 or 18.04 Operating System to make this work. To install everything properly I suggest you follow this part of the tutorial. You can also scroll back to previous parts to make sure everything is right. There I show how to install Anaconda and other dependencies to make sure everything works correctly. To be sure, try to start from here. The tensorflow-1.12.0-cp36-cp36m-linux_x86_64.whl file is the one that installs the Tensorflow.

I hope you understand! Good luck!

[mark2146]

If I understand correctly, you are using YOLO neural networks and then converting them into a TensorFlow format that can run on the PYNQ-Z2. But what I want to do now is to train my own neural network, for example using VGG19 with transfer learning, and then deploy it to the PYNQ-Z2. In that case, which version of TensorFlow should I use to train my neural network?

[andre1araujo]

Ah okay. Sorry for the confusion. You should indeed try to install Tensorflow 1.12 and train your model. The resulting model will be a ".pb" file that then can be used the same way I used on the project. I don't know the details to install a older version of Tensorflow. If you cannot install that specific version, try a older one. I think the compatible version of python is 3.6 so make sure to install it first.The installation package depends on your Operating System but you should find tutorials on how to install it correctly. If you are using anaconda you can do it just like I did on the tutorial. About Tensorflow, the installation should be something simple as: pip install tensorflow==1.12.0. After that you might need to install other packages like scikit-learn for example. You can find implementations of VGG19 and other neural networks in python and Tensorflow but make sure it is a older version as they changed quite a lot in Tensorflow 2.0.

[mark2146]

Thank you! I’ll continue to give it a try.

[andre1araujo]

Sure, If you end up having more simple issues that require a lot of communication, you can message me on Linkedin here. All this can be a bit overwhelming for someone with not that much experience so I am here to help!

[mark2146]

Can you provide me with the link again? The one I have leads to a 404 error.

[mark2146]

I have now trained my model. Should I proceed to quantize it and then deploy it onto the PYNQ-Z2 board? Since my model is already saved as a .pb file, can I skip the earlier steps? Also, do these steps need to be performed on Ubuntu, or can they be done on other operating systems? Could you please provide me with the Linked link again? I'm getting a 404 error on my end.Sorry for the continuous disturbance.

[andre1araujo]

Sorry for the late response. I don't knwo how to share the profile haha. Can you give me yours and then I add you?

You can proceed to the quantization part! Use your ".pb" file and also use the same dataset you used to train the model instead of the COCO dataset I used on the tutorial. After quantization and compilation you will have to modify the c++ script but I will tell you about that later. Good luck and thank you for your patience!

[mark2146]

Do you have any software I can contact you with? Because I don’t have LinkedIn.

[mark2146]

I sent a invite.My name is 798!