Is your feature request related to a problem? Please describe.
Currently, it is not possible to test an IceVision model with Pytorch Lightning (we can only train it but cannot test it).
Describe alternatives you've considered
I've considered using the trainer.validate since IceVision already overrides the validation_step but this is not ideal since the validation step logs using self.log("val_loss", loss) which means that test metrics would be grouped with validaton metrics (eg: Using wandb logging).
This is not ideal since we lose the ability to clearly distinct validation metrics from test metrics.
Additional context
PL Error :
MisconfigurationException: No `test_step()` method defined to run `Trainer.test`.
🚀 Feature
Is your feature request related to a problem? Please describe. Currently, it is not possible to test an IceVision model with Pytorch Lightning (we can only train it but cannot test it).
Describe the solution you'd like We should override the
test_stepmethod in the IceVision Pytorch Lightning Model Adapter class so that it can be used by the Pytorch Lightning Trainer class (see https://pytorch-lightning.readthedocs.io/en/stable/common/trainer.html#testing )Describe alternatives you've considered I've considered using the
trainer.validatesince IceVision already overrides thevalidation_stepbut this is not ideal since the validation step logs usingself.log("val_loss", loss)which means that test metrics would be grouped with validaton metrics (eg: Using wandb logging).This is not ideal since we lose the ability to clearly distinct validation metrics from test metrics.
Additional context PL Error :