Closed fjremnav closed 2 years ago
Hi @fjremnav,
We'll update dataset for multi classes soon.
In order to use multi-classes, you need to edit hybridnets/dataset.py
:
# Splitting traffic lights into 4 colors
self.id_dict = {'traffic sign': 0, 'tl_green': 1, 'tl_red': 2,
'tl_yellow': 3, 'tl_none': 4, 'person': 5}
# Or if you just want to detect traffic lights with no color distinction, change to
self.id_dict = {'traffic sign': 0, 'traffic light': 1, 'person': 2}
# and comment these lines out:
# if category == "traffic light":
# color = obj['attributes']['trafficLightColor']
# category = "tl_" + color
and projects/project.yml
:
# With color
obj_list: ['traffic sign', 'tl_green', 'tl_red', 'tl_yellow', 'tl_none', 'person']
# No color
obj_list: ['traffic sign', 'traffic light', 'person']
There are 2 variables, self.id_dict and self.id_dict_single in the dataset.py and which one I need to change: self.id_dict = {'person': 0, 'rider': 1, 'car': 2, 'bus': 3, 'truck': 4, 'bike': 5, 'motor': 6, 'tl_green': 7, 'tl_red': 8, 'tl_yellow': 9, 'tl_none': 10, 'traffic sign': 11, 'train': 12} self.id_dict_single = {'car': 0, 'bus': 1, 'truck': 2, 'train': 3}
Thanks,
Thank you for your questions. We've realized that it's too troublesome to do manually and so we changed them a little bit.
Now to do multi-class, you only have to edit project.yml
from:
obj_list: ['car']
obj_combine: ['car', 'bus', 'truck', 'train'] # if single class, combine these classes into 1 single class in obj_list
# leave as empty list ([]) to not combine classes
to:
obj_list: ['person', 'traffic sign', 'traffic light']
obj_combine: []
Happy training :smile:
I change project.yaml according to your suggestion:
obj_list: ['person', 'traffic sign', 'traffic light'] obj_combine: []
After finishing the training and run inference using: python hybridnets_test.py -w checkpoints/None/hybridnets-d3_0_17500.pth --source demo/image --output demo_result --imshow False --imwrite True
I got the following errors:
self.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for HybridNetsBackbone: size mismatch for classifier.header.pointwise_conv.conv.weight: copying a param with shape torch.Size([27, 160, 1, 1]) from checkpoint, the shape in current model is torch.Size([9, 160, 1, 1]). size mismatch for classifier.header.pointwise_conv.conv.bias: copying a param with shape torch.Size([27]) from checkpoint, the shape in current model is torch.Size([9]).
What else do I need to change to run tests?
We forgor 💀
Please pull the latest code and run inference again, should be working fine now.
I pull the latest code and run it again. Now I have a different error below:
Traceback (most recent call last):
File "hybridnets_test.py", line 105, in
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "hybridnets_test.py", line 107, in
anchor_ratios = params.anchors_scales
anchor_scales = params.anchors_ratios
We did a little oopsie haha
I pull the latest code and test seems working OK. However, quality is quite bad on test images with my trained model using bdd100k dataset.
See the attachment of inferencing result of 4.jpg
Do you have a trained weight with traffic light, traffic sign and car in the detection classes?
Hi @fjremnav, before training your network. You need to recalculate the anchor scales and anchor ratios and modify your anchor scale in backbone.py. Secondly, training object detection task is a challenge, it took at least 250-300 epochs to model can convergence.
@datvuthanh
With the latest code, I am not able to do training after 1st. Below is the log: Step: 17499. Epoch: 0/500. Iteration: 17500/17500. Cls loss: 0.76407. Reg loss: 7.51139. Seg loss: 0.48101. Total loss: 8.75646: 100%|██████████| 17500/17500 [1:32:22<00:00, 3.16it/s] checkpoint... checkpoint... checkpoint... checkpoint... checkpoint... checkpoint... checkpoint... checkpoint... checkpoint... checkpoint... checkpoint... checkpoint... checkpoint... checkpoint... checkpoint... checkpoint... checkpoint... checkpoint... checkpoint... checkpoint... checkpoint... checkpoint... checkpoint... checkpoint... checkpoint... checkpoint... checkpoint... checkpoint... checkpoint... checkpoint... checkpoint... checkpoint... checkpoint... checkpoint... checkpoint... 100%|██████████| 2500/2500 [14:49<00:00, 2.81it/s]Killed
Have you successfully trained with traffic lights/signs and pedetstrian with decent results?
Hi @fjremnav, we did train them for a few epochs and it worked fine. Let's look at your log:
=> This is because we set nms_threshold at 0.001 to get the best recall possible in our paper. So when calculating metrics, it has to handle a large amount of bounding boxes, leading to out-of-memory, and finally exploding the program before the next epoch.
That being said, there are multiple ways to circumvent this problem, choose the best that suit you:
car
class with 64GB RAM.python train.py --cal_map False
to not calculate metrics when validating. This option will only print validation losses. When the losses seem to flatten and the weather is nice, rent a high-RAM instance to validate the best weight with python val.py -w checkpoints/xxx_best.pth
. We actually did this to save on cost.val.py
to something reasonable like 0.5, depending on your application and end goals. You don't have to get best recall unless you're either helping us by experimenting 🥰 or competing with us 😠.Got that.
How to modify bdd100k.yaml so object detection includes traffic sign, traffic light and pedestrian to detection?
Thanks,