Training for federated model

[atusi-nakajima]

I have a question about Training for federated models. I don't understand the difference between the Custom Dataset and the distributed Training for federated model. Am I correct in assuming that the Training for federated model in the readme link is the KITTI format data that you created in Custom Dataset Generation? Thank you very much for your help.

[iamhyc]

@bearswang Please help answer this question, thanks.

[bearswang]

First, the command python3 gen_data/Scenario.py record [x_1,...x_32] [y_1,...y_5] only generates raw data (not KITTI format). The raw data folder is something like raw_data/record2020_xxxx_xxxx

Second, to generate a customized dataset with KITTI format, further run the command python3 gen_data/Process.py raw_data/record2020_xxxx_xxxx

This will create a folder $ROOT_PATH/dataset/record2020_xxxx_xxxx It looks like:

dataset
+- record2020_xxxx_xxxx
      +- global_label   #global labels
      +- vhicle.xxx.xxx_xxx
          +- calib00
              +- 0000.txt
              +- 0001.txt
          +- image00
              +- 0000.jpg
              +- 0001.jpg
          +- label00
              +- 0000.txt
              +- 0001.txt
          +- velodyne
              +- 0000.bin
              +- 0001.bin
      +- vhicle.xxx.xxx_xxx
      +- img_list.txt   # frame IDs

Finally, our federated training is based on PCDet project. The data format is slightly different from the above (kindly see the PCDet readme). The dataset folder looks like:

data
   +- record2020_xxxx_xxxx
      +- global_label    # same as global_label in “$ROOT_PATH/dataset/record2020_xxxx_xxxx”
      +- vhicle.xxx.xxx_xxx
   +- Imagesets
      +- train.txt   # same format as img_list.txt in “$ROOT_PATH/dataset/record2020_xxxx_xxxx”
      +- test.txt
      +- val.txt
   +- training
      +- calib  # same as calib00 in “$ROOT_PATH/dataset/record2020_xxxx_xxxx”
          +- 0000.txt
          +- 0001.txt
      +- image_2   # same as image00 in “$ROOT_PATH/dataset/record2020_xxxx_xxxx”
          +- 0000.jpg
          +1 0001.jpg
      +- label_2   # same as label00 in “$ROOT_PATH/dataset/record2020_xxxx_xxxx”
          +- 0000.txt
          +- 0001.txt
      +- velodyne   # same as velodyne in “$ROOT_PATH/dataset/record2020_xxxx_xxxx”
          +- 0000.bin
          +- 0001.bin

With the above data structure, run the following: cd ./PCDet/pcdet/datasets/kitti python3 preprocess.py create_kitti_infos record2020_xxxx_xxxx vhicle_id This will create several info files for training and testing in PCDet. More details can be found at: https://github.com/open-mmlab/OpenPCDet