AxisCommunications / acap-native-sdk-examples

Example code for APIs and features in AXIS Camera Application Platform (ACAP) Native SDK
Apache License 2.0
44 stars 25 forks source link

Failure to load Mediapipe's Face landmark model on DLPU #301

Open Poufy opened 1 month ago

Poufy commented 1 month ago

We are trying to load the Mediapipe's Face landmark model on the DLPU using the Larod API but we fail to do so with the following error:

image

And running journalctl

image

We are however able to load different models like Mediapipe's Face detection model

To reproduce

We're following the same setup as the vdo-larod example, however, we're using CPP. Below is the function where we attempt to load the model

bool setupLarod(const char *modelFile, larodConnection **conn, larodModel **model)
{
    larodError *error = NULL;
    *conn = NULL;
    *model = NULL;
    int larodModelFd = -1;
    const larodDevice *dev = NULL;

    if (!larodConnect(conn, &error))
    {
        std::cout << "Could not connect to larod: " << error->msg << std::endl;
        goto error;
    }

    larodModelFd = open(modelFile, O_RDONLY);
    if (larodModelFd < 0)
    {
        std::cout << "Unable to open model file: " << modelFile << std::endl;
        goto error;
    }

    dev = larodGetDevice(*conn, "axis-a8-dlpu-tflite", 0, &error);
    if (!dev)
    {
        std::cout << "Unable to get device: " << error->msg << std::endl;
        goto error;
    }

    *model = larodLoadModel(*conn, larodModelFd, dev, LAROD_ACCESS_PRIVATE, "", NULL, &error);
    if (!*model)
    {
        std::cout << "Unable to load model: " << error->msg << std::endl;
        goto error;
    }

    close(larodModelFd);
    return true;

error:
    if (larodModelFd >= 0)
    {
        close(larodModelFd);
    }
    if (*conn)
    {
        larodDisconnect(conn, NULL);
    }
    larodClearError(&error);
    return false;
}

And we call it like so

        char modelFile[] = "/usr/local/packages/opencv_app/face_landmark.tflite";
        if (!setupLarod(modelFile, &larodConn, &model))
        {
            std::cout << "setupLarod failed" << std::endl;
            exit(1);
        }

Environment

Additional context

When swapping the device to use the CPU instead, the model is loaded without any issues.

Is this a hardware restriction? Is that operations in the model aren't supported by the DLPU?

pataxis commented 1 month ago

Hi @Poufy , thanks for reaching out with your question.

We have a good documentation on computer vision with many details, please have a look there first if it can answer your question.