how to evaluate/test mae or multimae on test dataset?

[ucalyptus2]

@dmizr @amir32002 @roman-bachmann thanks for the paper

is there a way in the code already to perform testing on a different dataset or is it required to code it up ourselves?

[dmizr]

Hi @forkbabu ,

If you are performing a standard classification task, you should be able to use the existing run_finetuning_cls.py script to run evaluation on a different dataset. It's worth noting that this has only been evaluated on ImageNet in our paper, but you should be able to change the config (e.g., nb_classes) and augmentations to make it work with your dataset. You can take a look at the evaluate function for more details.

If your task isn't classification, you will need to implement the evaluation yourself if you aren't using one of the datasets that we reported on in the paper.

[kalyani7195]

Thanks for the great work, btw! Any hints on inference(no fine-tuning) on the eval dataset for NYU val set? Thanks!