search

voldemortX / pytorch-auto-drive

PytorchAutoDrive: Segmentation models (ERFNet, ENet, DeepLab, FCN...) and Lane detection models (SCNN, RESA, LSTR, LaneATT, BézierLaneNet...) based on PyTorch with fast training, visualization, benchmarking & deployment help
BSD 3-Clause "New" or "Revised" License
850 stars 138 forks source link

Why is the BN layer not added before the softmax function? #51

Closed daigang896 closed 2 years ago

daigang896 commented 2 years ago

Hello： pytorch-auto-drive/torchvision_models/common_models.py 341 line: output = F.softmax(output, dim=1)。

Hello, why didn't you add the BN layer before the softmax function? In addition, does this "class EDLaneExist(nn.Module) " have any optimization space when the maximum number of lanes exceeds 6?

voldemortX commented 2 years ago

@daigang896 Actually if you are considering optimization, the problem may be from the 5 set here. But the effects seem minor on current results (only some 6 lanes on TuSimple) so we kept it for BC.

What do you mean by BN after softmax, is it a common practice? The intuition here is to infer existence based on pixel-wise probability map.

daigang896 commented 2 years ago

@voldemortX Hello, I'm just adding BN layer before this softmax, I'm not sure if it's useful. Trying to see if it works. “pytorch-auto-drive/blob/master/torchvision models/common models. py:319: self.layers_ final. append(nn.Conv2d(32, 5, (1, 1), stride=1, padding=(0, 0), bias=True))". I set the number 5 a little bigger. I understand the intention here. I'm just thinking about how to improve the lane line accuracy a little more.

voldemortX commented 2 years ago

@daigang896 Sounds interesting, please inform me here if it works.

daigang896 commented 2 years ago

@voldemortX OK.

daigang896 commented 2 years ago

@voldemortX It can not improve the accuracy of lane line detection. Maybe erfnet_SCNN or erfnet_RESA can only achieve a certain accuracy. The backbone network may need to be replaced.

voldemortX commented 2 years ago

@daigang896 Yes I agree that the bottleneck is the algorithms themselves. We are currently exploring alternative backbones such as RepVGG, MobileNet, and Transformers. Part of RepVGG might show up along with the upcoming refactoring #45 (it's scheduled for 12.31 but we might not make it, just stay tuned).

daigang896 commented 2 years ago

@voldemortX OK, look forward to new updates.

voldemortX commented 2 years ago

@daigang896 RepVGG, MobileNet, and Transformers are now supported (baselines and some RESA/SCNN). Furthermore, BézierLaneNet (CVPR 22) is integrated, which requires both keypoint and binary seg annotations (it should be easy to get them from keypoint annotations). However, we don't see a much higher performance on public datasets. You can try them out yourself now with the latest code.

I'll close this issue for now. Feel free to reopen or open a new one for further discussions.