search

valeoai / obow

Other
96 stars 17 forks source link

custom data #5

Closed hongjianyuan closed 3 years ago

hongjianyuan commented 3 years ago

How to use custom data？What needs to be modified? Thanks

gidariss commented 3 years ago

Hi @hongjianyuan,

In order to train with your custom data, you need to make the following changes on the code:

  1. Add an extra dataset_name case here https://github.com/valeoai/obow/blob/main/obow/datasets.py#L266 for your custom data. E.g.: 
    if dataset_name == "ImageNet":
dataset_train, dataset_test = get_ImageNet_data_for_obow(data_dir, **kwargs)
elif dataset_name == "MyCustomData":
dataset_train, dataset_test = get_MyCustom_data_for_obow(data_dir, **kwargs)
else:
raise NotImplementedError(f"Not supported dataset {dataset_name}")
  2. Implement the get_MyCustom_data_for_obow() function inside dataset.py that should be exactly like the existing get_ImageNet_data_for_obow() function with the only difference being that this point https://github.com/valeoai/obow/blob/main/obow/datasets.py#L238 you should load your own custom data.
  3. You should create an new config file like the config/ImageNetFull/ResNet50_OBoW_full.yaml (https://github.com/valeoai/obow/blob/main/config/ImageNetFull/ResNet50_OBoW_full.yaml) but: 3.a Leave the num_classes field empty here https://github.com/valeoai/obow/blob/main/config/ImageNetFull/ResNet50_OBoW_full.yaml#L18 3.b Specify in the dataset_name field your custom data https://github.com/valeoai/obow/blob/main/config/ImageNetFull/ResNet50_OBoW_full.yaml#L35