Odd behavior with prediction_from_dl

[Brainkite]

I've managed to train an mmdet Retinanet RN50 to a specific task with decent performances. But when I load the weights on a fresh model and a fresh kernel to perform just predictions, predictions are very poor compared to the trained and saved model, almost non of them is above det threshold. But for some reason, doing 1 epoch of training with the model and then loading the desired weights, resolves the issue and the model performs as expected.

This performs poor:

learn = model_type.fastai.learner(dls=[train_dl, valid_dl], model=model, metrics=metrics, path= root_dir) learn.load('Euc_APmed_469') infer_dl = model_type.infer_dl(valid_ds, batch_size=2, shuffle=False) preds = model_type.predict_from_dl(model, infer_dl, keep_images=True)

This performs properly:

learn = model_type.fastai.learner(dls=[train_dl, valid_dl], model=model, metrics=metrics, path= root_dir) learn.fit(1) learn.load('Euc_APmed_469') infer_dl = model_type.infer_dl(valid_ds, batch_size=2, shuffle=False) preds = model_type.predict_from_dl(model, infer_dl, keep_images=True)

Icevision 0.8.1 pytorch 1.8.0 torchvision 0.9.0 fastai 2.3.1 python 3.8.10 Ubuntu 18.04.5

[lgvaz]

That is very strange indeed. Can you provide a colab notebook training on a simple dataset (maybe fridge?) that reproduces the issue?

[Brainkite]

That is very strange indeed. Can you provide a colab notebook training on a simple dataset (maybe fridge?) that reproduces the issue?

Okay, finally took the time to reproduce it in one of your Colab Nbs: https://colab.research.google.com/drive/1MkvqCjobGyateYeOXsEZsc_xg5CyANrH?usp=sharing

The issue can be witnessed here: https://colab.research.google.com/drive/1MkvqCjobGyateYeOXsEZsc_xg5CyANrH?authuser=1#scrollTo=s8hnDq1WKgx0&line=1&uniqifier=1

[Brainkite]

@lgvaz Any update on this?

[FraPochetti]

@Brainkite does this issue persist even using icevision from master?

[FraPochetti]

Closing due to lack of activity.