All values are 0.0 in output

[azad96]

Hi, I've added the all the 5 grid sample files to my example project written in TensorRT C++ API. Then, I feed an input and grid that I initialized in my main.cpp to the plugin layer in order to see how it works as follows:

nvinfer1::ITensor* input = network->addInput("INPUT", DataType::kFLOAT, nvinfer1::Dims{4, {1, 1, 4, 4}});
nvinfer1::ITensor* grid = network->addInput("GRID", DataType::kFLOAT, nvinfer1::Dims{4, {1, 8, 8, 2}});
assert(input); assert(grid);

auto interpolation_mode = torch::detail::GridSamplerInterpolation::Bilinear;
auto padding_mode = torch::detail::GridSamplerPadding::Zeros;
auto align_corners = true;

auto creator = getPluginRegistry()->getPluginCreator("GridSampler", "1");
const nvinfer1::PluginFieldCollection* pluginData = creator->getFieldNames();

nvinfer1::PluginFieldCollection mPFC;
std::vector<nvinfer1::PluginField> mPluginAttributes;
mPluginAttributes.emplace_back(nvinfer1::PluginField("interpolation_mode", &interpolation_mode,
                                                    nvinfer1::PluginFieldType::kINT32, 1));
mPluginAttributes.emplace_back(nvinfer1::PluginField("padding_mode", &padding_mode,
                                                    nvinfer1::PluginFieldType::kINT32, 1));
mPluginAttributes.emplace_back(nvinfer1::PluginField("align_corners", &align_corners,
                                                    nvinfer1::PluginFieldType::kINT32, 1));
mPFC.nbFields = mPluginAttributes.size();
mPFC.fields = mPluginAttributes.data();

nvinfer1::IPluginV2 *pluginObj = creator->createPlugin("GridSampler", &mPFC);
nvinfer1::ITensor* inputTensors[] = {input, grid};
auto grid_sample_plugin = network->addPluginV2(inputTensors, 2, *pluginObj);

grid_sample_plugin->getOutput(0)->setName("OUTPUT");
network->markOutput(*grid_sample_plugin->getOutput(0));

The code works but all the values of output are 0. I feed the same input and grid in python, so I know it should not be 0. I am using TensorRT 6.0 and CUDA 10.1. Any idea what I am doing wrong?

[azad96]

After some debugging, I've found out that "mBatch" in the following function in gridSamplerPlugin.cpp is not set yet. Therefore, its value is 0, which results in the problem I stated above.

int GridSamplerPlugin::enqueue(int batchSize, const void*const * inputs, void** outputs, void* workspace, cudaStream_t stream)
{
    int status = -1;

    GridSamplerDataType dataType = (mType == DataType::kFLOAT ? GridSamplerDataType::GFLOAT : GridSamplerDataType::GHALF);

    status = grid_sampler_2d_cuda(mBatch, inputs[0], inputs[1], outputs[0],
                                mInputChannel, mInputHeight, mInputWidth, mGridHeight, mGridWidth,
                                mInputChannel*mInputHeight*mInputWidth, mInputHeight*mInputWidth, mInputWidth, 1,
                                mGridHeight*mGridWidth*2, mGridWidth*2, 2, 1,
                                mInputChannel*mGridHeight*mGridWidth, mGridHeight*mGridWidth, mGridWidth, 1,
                                mInterpolationMode, mPaddingMode, mAlignCorners, dataType, stream);

    return status;
}

Using enqueue's parameter batchSize instead of mBatch fixed the problem for me. I guess serializing mBatch member like others (mInputChannel, mInputHeight, etc.) will be another solution but mine seems easier.