This issue also includes the investigation into whether visual corrupts the dataset statistics. The dataset statistics is only calculated on the individual dataset as opposed to the ConcatDataset. However, the issue seems to be that we are using the ConcatDataset for validation and test. Therefore by removing that, we can remove the performance issue. In other words, maybe we can rerun the test performances for the older models?
… val and test.
This issue also includes the investigation into whether visual corrupts the dataset statistics. The dataset statistics is only calculated on the individual dataset as opposed to the ConcatDataset. However, the issue seems to be that we are using the ConcatDataset for validation and test. Therefore by removing that, we can remove the performance issue. In other words, maybe we can rerun the test performances for the older models?