kargarisaac commented 5 years ago

Hi, I want to train a single model for all classes (cars/pedestrians/cyclists or maybe trucks and vans). I changed the config file but when I run the code it says it cannot find some files: FileNotFoundError: /home/isaac/python/avod/avod/data/mini_batches/iou_2d/kitti/train/lidar/All[ 0.5 0.5 0.5]/000580.npy not found for sample 000580 in /home/isaac/python/avod/avod/data/mini_batches/iou_2d/kitti/train/lidar, run the preprocessing script first

I check the folder and found some folders such as cars[0.5], people[0.5, 0.5], but no all[0.5, 0.5, 0.5]. So I have two question:

Did you train a single network for all classes and the performance was bad and just don't want to release it? Cuz you mentioned different configs for your different networks for cars and people. I think because of different sizes cause these different configs.
If I want to train a single network, which files do I need to modify.

I think training one single network with a single config and pre-defined 3d anchors can cause lower accuracy and higher inference time. Correct me if I'm wrong, please.

Do you suggest to do this or not.

Thank you.

kargarisaac commented 5 years ago

Here is the config file I used: `

Example cars-people config with pyramid feature extractor and augmentation

For each experiment, copy this config and modify the copy

model_config { model_name: 'avod_model' checkpoint_name: 'pyramid_cars_people_with_aug_example'

input_config {
    bev_depth: 6
    img_depth: 3

    img_dims_w: 1200
    img_dims_h: 360
}

rpn_config {
    rpn_proposal_roi_crop_size: 3
    rpn_fusion_method: 'mean'
    rpn_train_nms_size: 1024
    rpn_test_nms_size: 1024
    rpn_nms_iou_thresh: 0.8
}

avod_config {
    avod_proposal_roi_crop_size: 7
    avod_positive_selection: 'not_bkg'
    avod_nms_size: 100
    avod_nms_iou_thresh: 0.01
    avod_box_representation: 'box_4ca'
}

label_smoothing_epsilon: 0.001
expand_proposals_xz: 0.0
# To disable path drop, set both to 1.0
path_drop_probabilities: [0.9, 0.9]
train_on_all_samples: False
eval_all_samples: False

layers_config {
    bev_feature_extractor {
        bev_vgg_pyr {
            vgg_conv1: [2, 32]
            vgg_conv2: [2, 64]
            vgg_conv3: [3, 128]
            vgg_conv4: [3, 256]
            l2_weight_decay: 0.0005
        }
    }
    img_feature_extractor {
        img_vgg_pyr {
            vgg_conv1: [2, 32]
            vgg_conv2: [2, 64]
            vgg_conv3: [3, 128]
            vgg_conv4: [3, 256]
            l2_weight_decay: 0.0005
        }
    }

    rpn_config {
        cls_fc6: 256
        cls_fc7: 256
        reg_fc6: 256
        reg_fc7: 256
        l2_weight_decay: 0.0005
        keep_prob: 0.5
    }
    avod_config {
        # basic_fc_layers {
        #     num_layers: 3
        #     layer_sizes: [2048, 2048, 2048]
        #     l2_weight_decay: 0.005
        #     keep_prob: 0.5
        #     fusion_method: 'mean'  # 'mean' or 'concat'
        # }
        fusion_fc_layers {
            num_layers: 3
            layer_sizes: [2048, 2048, 2048]
            l2_weight_decay: 0.005
            keep_prob: 0.5
            fusion_method: 'mean'  # 'mean', 'concat', or 'max'
            fusion_type: 'early'  # 'early', 'late', 'deep'
        }
    }
}

# Loss function weights
loss_config {
    cls_loss_weight: 1.0
    reg_loss_weight: 5.0
    ang_loss_weight: 1.0
}

}

train_config {

batch_size: 1

optimizer {
    adam_optimizer {
        learning_rate {
            exponential_decay_learning_rate {
                initial_learning_rate: 0.0001
                decay_steps: 100000
                decay_factor: 0.1
            }
        }
    }
}

overwrite_checkpoints: False

max_checkpoints_to_keep: 10000
max_iterations: 120000
checkpoint_interval: 1000

summary_interval: 10
summary_histograms: False
summary_img_images: False
summary_bev_images: False

allow_gpu_mem_growth: True

}

eval_config { eval_interval: 1000 eval_mode: 'val' ckpt_indices: -1 evaluate_repeatedly: True

allow_gpu_mem_growth: True

}

dataset_config { name: 'kitti'

# dataset_dir: '~/Kitti/object'
# data_split: 'train'

data_split_dir: 'training'
has_labels: True

cluster_split: 'train'
classes: ['Car', 'Pedestrian', 'Cyclist']
num_clusters: [2, 1, 1]

bev_source: 'lidar'
aug_list: ['flipping', 'pca_jitter']

kitti_utils_config {
    area_extents: [-40, 40, -5, 3, 0, 70]
    voxel_size: 0.1
    anchor_strides: [0.5, 0.5, 0.5, 0.5, 0.5, 0.5]
    density_threshold: 1

    bev_generator {
        slices {
            height_lo: -0.2
            height_hi: 2.3
            num_slices: 5
        }
    }

    mini_batch_config {
        density_threshold: 1

        rpn_config {
            iou_2d_thresholds {
               neg_iou_lo: 0.0
               neg_iou_hi: 0.3
               pos_iou_lo: 0.45
               pos_iou_hi: 1.0
            }
            # iou_3d_thresholds {
            #     neg_iou_lo: 0.0
            #     neg_iou_hi: 0.3
            #     pos_iou_lo: 0.4
            #     pos_iou_hi: 1.0
            # }

            mini_batch_size: 512
        }

        avod_config {
            iou_2d_thresholds {
                neg_iou_lo: 0.0
                neg_iou_hi: 0.5
                pos_iou_lo: 0.6
                pos_iou_hi: 1.0
            }

            mini_batch_size: 1024
        }
    }
}

}

In thegen_mini_batches.py` the car_class and people_class are True.

Fredrik00 commented 5 years ago

You need to create a preprocessing config, such as those in mb_preprocessing that includes the classes Car Pedestrian and Cyclist. After that you need to set up preprocessing in gen_mini_batches.py to process All using the config you set up. If you want more classes you will at least also have to modify kitti_dataset.py as it is hardcoded to name joint detection of Pedestrian and Cyclist as 'People' and Pedestrian, Cyclist and Car as 'All', but I haven't tested expanding this yet.

I have forked the repo and done some work of my own, you can see examples of the two first files I mentioned at https://github.com/Fredrik00/avod/blob/master/avod/configs/mb_preprocessing/test_all.config and https://github.com/Fredrik00/avod/blob/master/scripts/preprocessing/gen_mini_batches.py

Keep in mind that results on pedestrians and cyclists will not be very good out of the box, likely due to the poor balancing of the classes in the dataset.

Jiachenyin1 commented 5 years ago

Hi! I create multi_class trainning before gen_min_batch with car people and cyclists. when i val this model, only one class (car) is showing in result! I wodder if you have same issue,please tell me why ?

Jiachenyin1 commented 5 years ago

You need to create a preprocessing config, such as those in mb_preprocessing that includes the classes Car Pedestrian and Cyclist. After that you need to set up preprocessing in gen_mini_batches.py to process All using the config you set up. If you want more classes you will at least also have to modify kitti_dataset.py as it is hardcoded to name joint detection of Pedestrian and Cyclist as 'People' and Pedestrian, Cyclist and Car as 'All', but I haven't tested expanding this yet.

I have forked the repo and done some work of my own, you can see examples of the two first files I mentioned at https://github.com/Fredrik00/avod/blob/master/avod/configs/mb_preprocessing/test_all.config and https://github.com/Fredrik00/avod/blob/master/scripts/preprocessing/gen_mini_batches.py

Keep in mind that results on pedestrians and cyclists will not be very good out of the box, likely due to the poor balancing of the classes in the dataset.

Hi! I create multi_class trainning before gen_min_batch with car people and cyclists. when i val this model, only one class (car) is showing in result! I wodder if you have same issue,please tell me why ?

RMobina commented 2 years ago

Hi! I create multi_class trainning before gen_min_batch with car people and cyclists. when i val this model, only one class (car) is showing in result! I wodder if you have same issue,please tell me why ?

Hi, did you find the solution? I have the same problem. I would be grateful if shre with me if you know the solution. Thanks

Jiachenyin1 commented 2 years ago

亲~邮件已经收到啦，我会及时查看的

kujason / avod

One single model for all classes #124

Example cars-people config with pyramid feature extractor and augmentation

For each experiment, copy this config and modify the copy