SuperPoint tensorrt test with tensor size mismatch error

[myboyhood]

Hello! I am learning swarm_loop package. I try to use src/loop_tensorrt_test.cpp to verify using superpoint trt to detect in my image. However I got the error of tensor size mismatch.

Problem generation:

1. I have deployed https://github.com/wang-xinyu/tensorrtx/ and https://github.com/xiang-wuu/SuperPointPretrainedNetwork
2. using python gen_wts.py to generate superpoint_v1.wts
3. I notice /tensorrtx/superpoint/superpoint.cpp at line 19-20 as following:

   static const int INPUT_H = 120; // 208
   static const int INPUT_W = 160; // 400 

4. then using the following command to generate supernet.engine.

   ./supernet -s SuperPointPretrainedNetwork/superpoint_v1.wts

5. My test code is as following:

   
   #include "superpoint_tensorrt.h"
   using namespace Swarm;

int main(int argc, char* argv[]) { if (argc < 2) { return -1; }

std::cout << "Load Model from " << argv[1] << std::endl;

std::string engine_path(argv[1]);

SuperPointTensorRT sp_trt(engine_path, "", "",  160, 120, 0.012, true); //origin code

// SuperPointTensorRT sp_trt(engine_path, "", "", 400, 208, 0.012, true);

std::cout << "Load Model success" << std::endl << " Loading image " << argv[2] << std::endl;

cv::Mat img = cv::imread(argv[2]);
cv::resize(img, img, cv::Size(160, 120));// origin code

// cv::resize(img, img, cv::Size(400, 208));// my change std::vector local_desc; std::vector kps;

cv::Mat img_gray;
cv::cvtColor(img, img_gray, cv::COLOR_BGR2GRAY);
TicToc tic;
for (unsigned int i = 0; i < 100; i ++) {
    sp_trt.inference(img_gray, kps, local_desc);
}
double dt = tic.toc();

std::cout << "\nSuperpoint 100 takes" << dt << std::endl;

for(auto pt : kps) {
    cv::circle(img, pt, 1, cv::Scalar(255, 0, 0), -1);
}

cv::resize(img, img, cv::Size(), 4, 4);
cv::imshow("Image", img);
cv::waitKey(-1);

}

5. I run the following command to test
```sh
rosrun superglue_feature_track loop_tensorrt_test /home/wzy/vins_ws/src/super-glue-pretrained-network-cius/models/weights/supernet_160_120.engine /home/wzy/vins_ws/src/super-glue-pretrained-network-cius/assets/room/lab1.png

6. In order to get more info, I add cout in tensorrt_generic.cpp to get the following terminal output

Load Model from /home/wzy/vins_ws/src/super-glue-pretrained-network-cius/models/weights/supernet_160_120.engine
Trying to init TRT engine of SuperPointTensorRT/home/wzy/vins_ws/src/super-glue-pretrained-network-cius/models/weights/supernet_160_120.engine
Loading TRT Engine...
Loading Complete!
TensorRT binding index 0 name data
TensorRT binding index 1 name semi
TensorRT binding index 2 name desc
MaxBatchSize1
Tensorsemi bind to 1 dim 65 15 20 0
Tensordesc bind to 2 dim 256 15 20 0
name:semi
inputDims.nbDims: 3
inputDims.d[i]: 65
inputDims.d[i]: 15
inputDims.d[i]: 20
19500:19200
name:desc
inputDims.nbDims: 3
inputDims.d[i]: 256
inputDims.d[i]: 15
inputDims.d[i]: 20
76800:76800

7. it throw out error as loop_tensorrt_test: /home/wzy/vins_ws/src/super-glue-pretrained-network-cius/src/tensorrt_generic.cpp:90: bool Swarm::TensorRTInferenceGeneric::verifyEngine(): Assertionget3DTensorVolume4(m_Engine->getBindingDimensions(tensor.bindingIndex)) == tensor.volume && "Tensor volumes dont match between cfg and engine file \n"' failed`

I find semi 19500:19200 is not match, I find 19500 = 651520 and 19200 = 641520. I do not know how to fix this error. Hope some help!!

[myboyhood]

The error is mainly that the tensor size is not matched when supernet.engine load and init. When we multiply the three dims of semi, we get 19500 = 65 15 20, I check 65 is specified by SuperPoint officially. but the supernet.engine load with tensor.volume = 19200 (64 15 20)

I think the problem is whether 65 or 64 is right when I use supernet.engine, How can I fix this error? much appreciation for any help !