关于多GPU问题

[ZHEQIUSHUI]

我想请教一下 这里没看到和模型绑定的信息 那如果我实例了两个Openpose 每一个openpose设置的device不一样 会有什么影响么（主要是现在没有双GPU的机子来测试 老板又要我告诉他。。。拜托了）

void Trt::SetDevice(int device) {
    std::cout<<("warning: make sure save engine file match choosed device\n");
    CUDA_CHECK(cudaSetDevice(device));
}

int Trt::GetDevice() const { 
    int* device = nullptr; //NOTE: memory leaks here
    CUDA_CHECK(cudaGetDevice(device));
    if(device != nullptr) {
        return device[0];
    } else {
        std::cout<<("Get Device Error\n");
        return -1;
    }
}

[zerollzeng]

不会有影响的，根据TensorRT的官方文档：

Q: How do I use TensorRT on multiple GPUs? A: Each ICudaEngine object is bound to a specific GPU when it is instantiated, either by the builder or on deserialization. To select the GPU, use cudaSetDevice() before calling the builder or deserializing the engine. Each IExecutionContext is bound to the same GPU as the engine from which it was created. When calling execute() or enqueue(), ensure that the thread is associated with the correct device by calling cudaSetDevice() if necessary.

来源：https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#faq

[ZHEQIUSHUI]

那我是不是需要修改Trt()的构造函数，然后在反序列化engine的之前就用setdevice，这样的话就会吧这个engine绑定对应的gpu？

[zerollzeng]

构造函数源码不需要修改吧，创建类之后直接SetDivice就可以了

[ZHEQIUSHUI]

是否需要在CreateEngine之前SetDevice?例如以下代码

OpenPose::OpenPose(const std::string& prototxt, 
                    const std::string& caffeModel,
                    const std::string& saveEngine,
                    const std::vector<std::string>& outputBlobName,
                    const std::vector<std::vector<float>>& calibratorData,
                    int maxBatchSize,
                    int runMode) {
    mNet = new Trt();
    mNet->SetDevice(device);//set gpu？？？
    mNet->CreateEngine(prototxt, caffeModel, saveEngine, outputBlobName, calibratorData, maxBatchSize, runMode);
    MallocExtraMemory();
}

[zerollzeng]

这样就ok了

[ZHEQIUSHUI]

谢谢你！我把这个issue关了啊