Problems using ArmNN in multithreading for ssd_mobilenet_v1

[thielchris]

Hello,

I am using ArmNN on a i.MX8M Plus with the ArmNN version 21.02. The following problem occurs for the VsiNpu backend (NXP specific backend). The NXP support says there might be a problem with ArmNN, so I ask the question here.

I have a problem in executing the ssd_mobilenet_v1 model (https://tfhub.dev/tensorflow/lite-model/ssd_mobilenet_v1/1/default/1) in my own application. In my main application I create a object of the ArmNNEngine class (see below). The initializing steps are fine. Then I run the model (via ArmNNEngine::run method) in a seperate thread (trough std::thread, std::future::async, QThread. Everything the same problem). The output tensor looks fine for the label id, score and number of detected objects. Just the coordinates of the boxes seems to be wrong and looking like undefinded memory (e.g. values like 3018, -3018, ..., -4.300e-29, ...). This problem only occurs when I run the inference for the ssd_mobilenet_v1 model. For every other model everything looks fine. When I don't use threads, also everything is fine. The input for the run method comes from a camera live stream. The output is draw on the live frame and is displayed using QOpenGLWidget. If I use QWidget instead of QOpenGLWidget, the coordinates are correct. So only in this combination I get wrong results: Using Threads, Using QOpenGLWidget, Using ssd_mobilenet_v1 model.

Has somebody an idea if there is a known problem?

Thank you for your help ! Kind regards, Chris

class ArmNNEngine
{
private:
    mutable armnn::OutputTensors                     m_aOutputTensors;        
    mutable armnn::InputTensors                      m_aInputTensors;       
    armnn::IRuntimePtr                               m_paRuntime;            
    armnn::NetworkId                                 m_aNetID;                
    armnnTfLiteParser::BindingPointInfo              m_aInputBindingInfo;    
    std::vector<armnnTfLiteParser::BindingPointInfo> m_vaOutputBindingInfo; 

    armnn::InputTensors  MakeInputTensors(const std::pair<armnn::LayerBindingId,
                                                          armnn::TensorInfo>& input,
                                          const void* pInputTensorData);

    armnn::OutputTensors MakeOutputTensors(const std::pair<armnn::LayerBindingId,
                                                           armnn::TensorInfo>& output);

    armnn::OutputTensors MakeOutputTensors(const std::vector<std::pair<armnn::LayerBindingId,
                                                           armnn::TensorInfo>>& output);

public:

    ArmNNEngine(const uint8_t &uBackendToUse,
                const std::string &sModelPath,
                const tModelType &stModelType,
                const uint16_t &uSourceFrameWidth,
                const uint16_t &uSourceFrameHeight,
                const tColorFormat &stSourceColorFormat,
                const tDataType &stDestDataType,
                const std::vector<uint16_t> &vuDestTensorSize);

    virtual ~ArmNNEngine();

    void run(const void* pSourceData,
                  void* pDestData);
};

[morgolock]

Hi @christthi

Could you please share the code calling into ArmNNEngine? In particular the code for initialization and inference.

Just to clarify, the problem you experience seems to be: 1) Only specific to ssd_mobilenet_v1 2) Reproducible when initialization and inference is done in different threads regardless the threading API (cpp11,qt,et)

We have not experienced problems with ssd_mobilenet_v1. I'd suggest you update to 21.11 if possible.

If you provide the code illustrating the problem we could try to reproduce it.

Hope this helps.

[MikeJKelly]

I'm closing this due to inactivity, if you still need help with this then please create a new ticket, thanks!