Bin-ze / BEVFormer_segmentation_detection

Implemented BEVFormer support for BEV segmentation
Apache License 2.0
98 stars 9 forks source link

Can I import recorded road videos? Can a single lens achieve the effect shown on your homepage? #10

Closed Alex-fishred closed 10 months ago

Bin-ze commented 10 months ago

This result is obtained after training on the nusence dataset. The implementation logic of the segmentation part is to add a segmentation branch after the BEV feature map and use segmentation annotations centered on the self-vehicle within the perception range for supervision.

If you use a single image such as the front view, it may take some effort to work, but from the perspective of the BEVformer algorithm, I think it is possible. It seems that it can be achieved by just modifying some configurations, but the effect is difficult to evaluate without conducting experiments.

Alex-fishred commented 10 months ago

Sorry, I may not fully understand what you mean Now my requirement is to put the road video I recorded in advance (only the front view) into the BEVFormer model to generate a 3D frame and road segmentation From your answer, I understand that there is a chance that inputting only the front view will work, but I don't quite understand what you mean by "This result is obtained after training on the nusence dataset. "

My mother tongue is Chinese If your native language is also Chinese, you can answer in Chinese and maybe I will understand you better.

Bin-ze commented 10 months ago

如果使用我的预训练模型,输入你的前视图图像序列的话,应该无法得到我演示的效果,但是如果你重新使用该代码库进行训练,应该可以满足你的要求,但是需要更改一些配置,比如BEV感知区域的大小(只是前视图的话目前以自车为中心的BEV区域不满足要求),除此之外,我认为还需要更改数据读取方式以及算法内部的一些源码才可能重新进行训练

Alex-fishred commented 10 months ago

好的,似乎看起來是一件龐大的工程 有任何消息或問題我會再次開啟這個提問 謝謝你的建議