SCLBD / DeepfakeBench

A comprehensive benchmark of deepfake detection
Other
512 stars 67 forks source link

Why do you discard the remaining data of DeepFakeDetection? #111

Open Dongyu-Han opened 1 week ago

Dongyu-Han commented 1 week ago
def prepare_testing_data(config):
    def get_test_data_loader(config, test_name):
        # update the config dictionary with the specific testing dataset
        config = config.copy()  # create a copy of config to avoid altering the original one
        config['test_dataset'] = test_name  # specify the current test dataset
        if not config.get('dataset_type', None) == 'lrl':
            test_set = DeepfakeAbstractBaseDataset(
                    config=config,
                    mode='test',
            )
        else:
            test_set = LRLDataset(
                config=config,
                mode='test',
            )

        test_data_loader = \
            torch.utils.data.DataLoader(
                dataset=test_set,
                batch_size=config['test_batchSize'],
                shuffle=False,
                num_workers=int(config['workers']),
                collate_fn=test_set.collate_fn,
                **drop_last = (test_name=='DeepFakeDetection'),**
            )

        return test_data_loader

    test_data_loaders = {}
    for one_test_name in config['test_dataset']:
        test_data_loaders[one_test_name] = get_test_data_loader(config, one_test_name)
    return test_data_loaders

Why do you discard the remaining data of DeepFakeDetection? This brings about a mismatch in data size in the evaluation. for example: image

Ftgn-dpA commented 2 days ago

Will removing this solve the problem?