the detection result of faster rcnn is 'nan'

[guoqiang0148666]

Hi, l found a question that the detection result of faster rcnn is nan (bbox_delte_data, score are nan, but roi_data is ok), i ask how to solver the problem, thanks.

l am looking forward to your reply, any suggestion will be welcome.

Here is my prototxt file: testnet.txt

[RudyFoo]

我试了最新的faster_rcnn，使用VGG16_faster_rcnn.caffemodel和对应的.prototxt转换出的.tmfile，是能正确检测到object的。你再试试，确认一下你的模型和转换工具是正常的。

[guoqiang0148666]

@RudyFoo 感谢您的回复

faster_rcnn vgg16的实验结果：

FRCNN VGG16 detection results on arm development board with Tengine engine:

root@linaro-alip:~/Tengine/examples/build/faster_rcnn# ./FASTER_RCNN -p /home/linaro/Downloads/VGG16_faster_rcnn.prototxt -m /home/linaro/Downloads/VGG16_faster_rcnn_final.caffemodel -i /root/Tengine/tests/images/ssd_dog.jpg
/root/Tengine/examples/build/faster_rcnn/FASTER_RCNN
height width scale300,400,0.520833
--------------------------------------
repeat 1 times, avg time per run is 11631.7 ms
bicycle : 79.717 %
BOX:( 96.6737 , 141.499 ),( 596.481 , 485.373 )
car : 98.878 %
BOX:( 461.973 , 70.1222 ),( 699.822 , 165.374 )
dog : 92.452 %
BOX:( 89.2659 , 209.347 ),( 340.918 , 546.499 )
======================================
[DETECTED IMAGE SAVED]: Faster_Rcnn
======================================

我的模型(只有一个检测类别)的结果：

./FASTER_RCNN -p sqznet_main_up.pt -m sqznet_main.caffemodel -i /root/Tengine/tests/images/ssd_dog.jpg
/root/Tengine/examples/build/faster_rcnn/FASTER_RCNN
height width scale300,400,0.520833
--------------------------------------
repeat 1 times, avg time per run is 385.707 ms
aeroplane   : nan %
BOX:( nan , nan ),( nan , nan )
aeroplane   : nan %
BOX:( nan , nan ),( nan , nan )
aeroplane   : nan %
BOX:( nan , nan ),( nan , nan )
...
aeroplane   : nan %
BOX:( nan , nan ),( nan , nan )

检测结果是nan

模型转换

./convert_caffe_to_tm -p sqznet_main_up.pt -m sqznet_main.caffemodel -o test-model.tmfile
Create tengine model file done: test-model.tmfile

模型可以转换成功

tmfile模型测试

./test_tm ../../../examples/build/tengine_model/convert/test-model.tmfile 300
Segmentation fault

不知道怎么用tmfile格式的文件进行faster rcnn的测试，但是用上面的命令出现段错误。

附上我的tmfile格式的源文件的netron结构图（如果需要，我也可以上传模型源文件）