The landmark position is drafting

[iamlaobie]

I found some image is OK for view, but the similarity from other similar image is very small, I draw the landmark like this:

[XiaochaoGONG]

It looks like the landmarks are not correctly detected from the view. Do all the other images act like that ?

[iamlaobie]

@XiaochaoGONG No, the wrong landmark is more frequent when the face be moving.I guess the landmark face and the saved face is not identical.

[XiaochaoGONG]

It shouldn't be, I'll check that on the latest version.

[XiaochaoGONG]

Hi, I've test the example camera_web_server with the latest esp-who version with related esp-face, all work fine.

[iamlaobie]

@XiaochaoGONG Thanks for your working! My application is based on esp-who, but I make many changing, so I can't merge official esp-who project. I only upgraded the esp-face to version 0.4.0 at this afternoon, I don't check the result. it can fixed the landmark drafting bugs?

[iamlaobie]

The problem exists after upgrade to v0.4.0(commit:33f3e46e34d95527b469e7990d46ca95a6a534c3)

[XiaochaoGONG]

What type of image are you using as the origin input ? Jpeg or RGB ? It cannot detect the face of that pose. So the landmarks and the box are the result of the last image ?

[iamlaobie]

The main process:

        fb = esp_camera_fb_get();
        if (!fb)
        {
            ESP_LOGE(TAG, "Camera capture failed");
            vTaskDelay(2000 / portTICK_PERIOD_MS);
            continue;
        }

        image_matrix = dl_matrix3du_alloc(1, fb->width, fb->height, 3);
        if (!image_matrix)
        {
            ESP_LOGE(TAG, "dl_matrix3du_alloc failed");
            continue;
        }

        if(!fmt2rgb888(fb->buf, fb->len, fb->format, image_matrix->item))
        {
            ESP_LOGW(TAG, "fmt2rgb888 failed");
            dl_matrix3du_free(image_matrix);
            continue;
        }

        box_array_t *net_boxes = face_detect(image_matrix, &mtmn_config);
        if (net_boxes)
        {
            if (align_face(net_boxes, image_matrix, aligned_face) == ESP_OK)
            {
                real_face_id = get_face_id(aligned_face);
                if (real_face_id) {
                    gpio_set_level(GPIO_LED_RED, 1);
                }
            }

            draw_face_boxes(image_matrix, net_boxes);

            if(!fmt2jpg(image_matrix->item, fb->width*fb->height*3, fb->width, fb->height, PIXFORMAT_RGB888, 90, &_jpg_buf, &_jpg_buf_len))
            {
                ESP_LOGE(TAG, "fmt2jpg failed");
            }
        }
        else
        {
            _jpg_buf = fb->buf;
            _jpg_buf_len = fb->len;
        }
        dl_matrix3du_free(image_matrix);

        if(request){
            size_t hlen = snprintf((char *)part_buf, 64, _STREAM_PART, _jpg_buf_len);
            res = httpd_resp_send_chunk(request, (const char *)part_buf, hlen);
            if(res == ESP_OK){
                res = httpd_resp_send_chunk(request, (const char *)_jpg_buf, _jpg_buf_len);
            }
            if(res == ESP_OK){
                res = httpd_resp_send_chunk(request, _STREAM_BOUNDARY, strlen(_STREAM_BOUNDARY));
            }
        }

        if (real_face_id) {
            post_image_feature((char *)_jpg_buf, _jpg_buf_len, real_face_id, net_boxes->box[0].box_p, net_boxes->landmark[0].landmark_p);
            dl_matrix3d_free(real_face_id);
            real_face_id = NULL;
        } else {
            post_image((char *)_jpg_buf, _jpg_buf_len);
        }
        if(fb){
            esp_camera_fb_return(fb);
            fb = NULL;
        } 
        if(_jpg_buf){
            free(_jpg_buf);
            _jpg_buf = NULL;
        }
        if (net_boxes) {
            free(net_boxes->box);
            free(net_boxes->landmark);
            free(net_boxes);
        }

if get_face_id return 512 dimensions vector, invoke post_image_feature function send data to server.

 post_image_feature((char *)_jpg_buf, _jpg_buf_len, real_face_id, net_boxes->box[0].box_p, net_boxes->landmark[0].landmark_p);

I draw landmark in server and show in browser like you see.

[XiaochaoGONG]

You can try to draw the landmarks right in the original image, just modify a little in draw_face_boxes (I remember that landmarks drawing is just commented). To test if the landmarks for its corresponding image are right.

[iamlaobie]

The landmark drafting problem is improved after upgrade to v0.4.0, but occur occasionally.

[XiaochaoGONG]

Hi, is the problem still there ?