A-Light-and-Fast-Face-Detector-for-Edge-Devices

[krishnak]

I notice that you have a fork of the project (A-Light-and-Fast-Face-Detector-for-Edge-Devices), is your implementation of Ultra light fast generic face detector an implementation of that paper. I am trying to train a head detection model which seems to have been already done in A-Light-and-Fast-Face-Detector-for-Edge-Devices, however it is in MXNET model, the mxnet to ONNX conversion fails with some slice error - appears to be a bug

https://github.com/onnx/onnx/issues/2245

I wanted to know whether you were able to convert the MXNET to ONNX model without any isues.

If your model is different from the A-Light-and-Fast-Face-Detector-for-Edge-Devices, can you let me know whether it can be used for transfer learning for head detection. Can you provide me with some pointers of converting your model to tensorflow for transfer learning

[Linzaer]

1. These two projects are different algorithms.
2. You can modify this project to train a head detection model.
3. I haven't converted this model to tf yet.
4. I haven't converted the mxnet model of A-Light-and-Fast-Face-Detector-for-Edge-Devices to onnx yet.

[krishnak]

Many thanks for your reply, I am trying to run this model on a Kendryte k210 chip, some one has ported your model to that chip. I will check out whether there is any performance degrade in the conversion process.https://github.com/kendryte/nncase/tree/master/examples/fast_facedetect

The kendryte model converter expects a caffe model or a tensorflow model.

I will first try to convert your current model to Kendryte model before venturing on retraining/modifying.

[ghost]

Many thanks for your reply, I am trying to run this model on a Kendryte k210 chip, some one has ported your model to that chip. I will check out whether there is any performance degrade in the conversion process.https://github.com/kendryte/nncase/tree/master/examples/fast_facedetect

The kendryte model converter expects a caffe model or a tensorflow model.

I will first try to convert your current model to Kendryte model before venturing on retraining/modifying.

@krishnak Maybe it should be reduce the number of priors and change the size of the feature map , which I have trained on the brainwash dataset. But trying to implement on tensorflow, there are still some problems on my k210 board.

[krishnak]

@xautzxc I noticed that you have reported a bug on Kendrte k233.

Are you able to run the fast face detect model on the Sipeed board? I am still setting up the development environment for Kendryte SDK. If you have compiled the fast face detect for Sipeed Maix Dock, can you upload it?

[ghost]

@krishnak In the nncase example, you can directly burn it to the sipeed board.

[krishnak]

@xautzxc Do you mean to say face.bin can run on sipeed board? - How is it possible? is the LCD driver same for both Sipeed and Kendryte boards? I thought Kendryte board has a 3.5'' LCD and Sipeed 2.4'' LCD

Edit: I tried to build the code using platform io and Kendryte standalone SDK, I am getting a linker error which I am trying to resolve. I solved the linker error by adding void * __dso_handle = 0; as a global variable in ultra_face.cpp.

Still having few issues in getting the image displayed

[ghost]

@krishnak I tested nncase examples on the maixduino board before and did not encounter the problem you said. I will test the examples on the maix dock board next week . All resolution is 320 x 240,though LCD have different physical size.

[krishnak]

@xautzxc Thanks for keeping up this conversation, I did a fresh download and compile just now, the following is the status on Sipeed Maix Dock when compiling using the platform io build setup.

1. Linker error as described above - it might disappear by using the correct linking flags, but I used the solution as above.
2. The LCD driver in the codebase seems to work for the 2.4'' screen however the image is pixelated - please see this https://forum.kendryte.com/topic/438/model-init-error-nncase-example/3 3.There is a Model init error so KPU inference doesn't work with incbin.
3. I recompiled the code (separately) with st7789.c which some one was using on a Maixdock - however the image is still pixelated. I have posted the link on the kendryte forum the same image as a JPEG using loboris micropython firmware. It appears that some information is lost during the image2c conversion

If you have a solution for the model init error let me know.

I will try to compile the other examples for nncase to see whether they work.

EDIT: I tried to install the kfpkg file which is provided with the iris example, I get a core dump with the following messages [INFO] Writing iris.kmodel to Flash address 0x00c00000 [INFO] Flashing data at Flash address 0x00c00000 Programming BIN: |======================================================================================| 100.0% 10kiB/s [INFO] Rebooting...

--[ MicroPython terminal ver. 5.1.3 ]-- --[ Press ESC twice for command mode ]--

flash init Cannot load kmodel. W (107341330) SYSCALL: sys_exit called by core 0 with 0x1

Yolo example: Again model init error. I think the incbin doesn't seem to work. I will try to use kfpkg to rebuild these examples and update

Further edit: I have figured out the reason for the model init error, I am using platformIO for building, it is using a SDK version which doesn't support nncase kmodel version greater than 3.

The incbin is supported only in the latest SDK of Kendryte, I will update my build setup and try it again

[krishnak]

I have this working on the Sipeed board, you need latest SDK for the nncase model to work, depending on which build setup one has chosen, it needs to be addressed.

Now to the results, the inference time is around 855 ms for a single frame. This is the time taken by the model to be executed in the KPU for infering the image. Image rendering and box drawing times are NOT included.

If the images are not exactly 320x240 the image2c is not padding it properly hence the result is garbage

[ghost]

@krishnak Congratulations. You can modify the model parameters to adapt your k210 to achieve higher frame rates(eg.5-6fps). BTW,Have you implement this method on tensorflow? And which dataset is used in yours?

[krishnak]

@xautzxc I need a model which needs to infer with in 20-30ms, the yoloV2 sipeed version is doing in camera inference around that, however that model is heavily pruned, it is detecting faces only of a certain size, I will do a comparison of yolov2 sipeed model with the images in this project to see how it performs i.e whether it detects all the faces as this one does.

Tensor flow - Not yet, I will first need to do this comparison with yolov2 before going there.

[krishnak]

@xautzxc I have run a comparison between the Sipeed yolov2 and this model on K210. Sipeed yolov2 has only 23 layers and all of them use a KL_K210_CONV which as per this link might be hardware accelerated. https://github.com/laanwj/k210-sdk-stuff/blob/master/doc/kpu.md Total time taken is 13ms.

Where as this face detector has 102 layers, and it uses 3 quantized Convulutions - each of them takes between 200-230ms that itself results in 660ms, there is one quantized binary taking 70ms the remaining layers only take 120ms in total.

I am yet to compare the accuracy of the yolov2 against this model, I am sure this model will be far superior in terms of accuracy.

[krishnak]

@xautzxc just figured out that the Slim model has been ported to k210 kmodel, it is having 73 layers and performs at 67ms on k210. Performance is in par with RFB-320 when I tested it with image 1.jpg with score thresholds at 0.15 and 0.75

@Linzaer Great work