search

yosungho / LineTR

Line as a Visual Sentence: Context-aware Line Descriptor for Visual Localization (Line Transformer)
Other
233 stars 35 forks source link

about fps #14

Closed yiyihao2000 closed 5 months ago

yiyihao2000 commented 1 year ago

Hello! Your work is very good, I personally have a question for you, how much is the demo running real-time fps? Can it be real-time? What configuration are you running the model on your computer? Looking forward to your answer, thank you very much!

yosungho commented 1 year ago

Hi, I just measured the computing time in my laptop (i3-1315U cpu) The total time was about 580 msec (1.7 FPS).

This number will be enhanced when you have a better GPU and/or TensorRT for the inference. You might want to change the backbone to be lighter.

yiyihao2000 commented 1 year ago

Hi, I just measured the computing time in my laptop (i3-1315U cpu) The total time was about 580 msec (1.7 FPS).

  • SuperPoint (backbone): 440 msec
  • LSD (detector): 23 msec
  • preprocessing: 26 msec
  • inference (LineTR): 53msec
  • matching: 1msec

This number will be enhanced when you have a better GPU and/or TensorRT for the inference. You might want to change the backbone to be lighter.

Thank you for your answer. At the same time, you explained in Part E of Chapter 3 of your paper that "Our networks Contain 14M parameters, and they run at an average speed Of 20 ms on an NVIDIA GTX 2080Ti GPU for 256 line Descriptors in an image Is 20ms the time spent on the individual inference process (LineTR) or the entire algorithm process? Looking forward to your answer, thank you very much!

yosungho commented 1 year ago

As far as I remember, the network speed mentioned in the paper reflects the inference time of the Superpoint+LineTR, but excludes the LSD (line segment detector) and preprocessing which are computed on the CPU. I understand this might have caused some confusion among readers.

The main intent was to describe that the Superpoint+LineTR network itself is inherently lightweight. At the time of writing the paper, my view was that CPU computation could be further optimized through methods such as C++ implementation, decreasing CPU and GPU conversions, or parallelization.