Inference after Finetuning

[mcdulltii]

May I know the way to use the .pt files after finetuning is performed on a dataset? I would like to load the .pt file and run inference on a single image, then print out the classification result from it, i.e. the classname.

Below is the command and output of my finetuning.

python -m src.wise_ft                                               \
    --train-dataset=RN50                                            \
    --epochs=20                                                     \
    --lr=0.00003                                                    \
    --batch-size=32                                                 \
    --cache-dir=cache                                               \
    --model=RN50                                                    \
    --eval-datasets=RN50                                            \
    --classnames="['true', 'false']" \
    --template=custom_dataset_template                              \
    --results-db=results.jsonl                                      \
    --save=models/wiseft/custom                                     \
    --data-location=/content/dataset                                \
    --alpha 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

Getting zeroshot weights.
100% 48/48 [00:01<00:00, 34.05it/s]
Saving image classifier to models/wiseft/custom/zeroshot.pt
Loading image classifier from models/wiseft/custom/zeroshot.pt
Fine-tuning end-to-end
Using devices [0]
Train Epoch: 0 [0% 0/21]    Loss: 3.895792  Data (t) 0.025  Batch (t) 2.184
Saving model to models/wiseft/custom/finetuned/checkpoint_1.pt
Saving image classifier to models/wiseft/custom/finetuned/checkpoint_1.pt
Evaluating on RN50
RN50 Top-1 accuracy: 0.1944
Results saved to results.jsonl.
Train Epoch: 1 [0% 0/21]    Loss: 3.722734  Data (t) 0.097  Batch (t) 0.668
Saving model to models/wiseft/custom/finetuned/checkpoint_2.pt
Saving image classifier to models/wiseft/custom/finetuned/checkpoint_2.pt
Evaluating on RN50
RN50 Top-1 accuracy: 0.4444
Results saved to results.jsonl.
Train Epoch: 2 [0% 0/21]    Loss: 3.029700  Data (t) 0.103  Batch (t) 1.164
Saving model to models/wiseft/custom/finetuned/checkpoint_3.pt
Saving image classifier to models/wiseft/custom/finetuned/checkpoint_3.pt
Evaluating on RN50
RN50 Top-1 accuracy: 0.5417
Results saved to results.jsonl.
Train Epoch: 3 [0% 0/21]    Loss: 2.278040  Data (t) 0.104  Batch (t) 1.193
Saving model to models/wiseft/custom/finetuned/checkpoint_4.pt
Saving image classifier to models/wiseft/custom/finetuned/checkpoint_4.pt
Evaluating on RN50
RN50 Top-1 accuracy: 0.5694
Results saved to results.jsonl.
Train Epoch: 4 [0% 0/21]    Loss: 1.359167  Data (t) 0.205  Batch (t) 0.938
Saving model to models/wiseft/custom/finetuned/checkpoint_5.pt
Saving image classifier to models/wiseft/custom/finetuned/checkpoint_5.pt
Evaluating on RN50
RN50 Top-1 accuracy: 0.6250
Results saved to results.jsonl.
Train Epoch: 5 [0% 0/21]    Loss: 1.342139  Data (t) 0.096  Batch (t) 1.317
Saving model to models/wiseft/custom/finetuned/checkpoint_6.pt
Saving image classifier to models/wiseft/custom/finetuned/checkpoint_6.pt
Evaluating on RN50
RN50 Top-1 accuracy: 0.6389
Results saved to results.jsonl.
Train Epoch: 6 [0% 0/21]    Loss: 1.333306  Data (t) 0.091  Batch (t) 1.026
Saving model to models/wiseft/custom/finetuned/checkpoint_7.pt
Saving image classifier to models/wiseft/custom/finetuned/checkpoint_7.pt
Evaluating on RN50
RN50 Top-1 accuracy: 0.7083
Results saved to results.jsonl.
Train Epoch: 7 [0% 0/21]    Loss: 0.917647  Data (t) 0.084  Batch (t) 1.141
Saving model to models/wiseft/custom/finetuned/checkpoint_8.pt
Saving image classifier to models/wiseft/custom/finetuned/checkpoint_8.pt
Evaluating on RN50
RN50 Top-1 accuracy: 0.8333
Results saved to results.jsonl.
Train Epoch: 8 [0% 0/21]    Loss: 0.812102  Data (t) 0.099  Batch (t) 0.603
Saving model to models/wiseft/custom/finetuned/checkpoint_9.pt
Saving image classifier to models/wiseft/custom/finetuned/checkpoint_9.pt
Evaluating on RN50
RN50 Top-1 accuracy: 0.7917
Results saved to results.jsonl.
Train Epoch: 9 [0% 0/21]    Loss: 1.104869  Data (t) 0.148  Batch (t) 0.852
Saving model to models/wiseft/custom/finetuned/checkpoint_10.pt
Saving image classifier to models/wiseft/custom/finetuned/checkpoint_10.pt
Evaluating on RN50
RN50 Top-1 accuracy: 0.8472
Results saved to results.jsonl.
Train Epoch: 10 [0% 0/21]   Loss: 0.608275  Data (t) 0.205  Batch (t) 0.761
Saving model to models/wiseft/custom/finetuned/checkpoint_11.pt
Saving image classifier to models/wiseft/custom/finetuned/checkpoint_11.pt
Evaluating on RN50
RN50 Top-1 accuracy: 0.8611
Results saved to results.jsonl.
Train Epoch: 11 [0% 0/21]   Loss: 0.696144  Data (t) 0.096  Batch (t) 0.904
Saving model to models/wiseft/custom/finetuned/checkpoint_12.pt
Saving image classifier to models/wiseft/custom/finetuned/checkpoint_12.pt
Evaluating on RN50
RN50 Top-1 accuracy: 0.8194
Results saved to results.jsonl.
Train Epoch: 12 [0% 0/21]   Loss: 0.845205  Data (t) 0.165  Batch (t) 1.253
Saving model to models/wiseft/custom/finetuned/checkpoint_13.pt
Saving image classifier to models/wiseft/custom/finetuned/checkpoint_13.pt
Evaluating on RN50
RN50 Top-1 accuracy: 0.8611
Results saved to results.jsonl.
Train Epoch: 13 [0% 0/21]   Loss: 0.884099  Data (t) 0.147  Batch (t) 0.730
Saving model to models/wiseft/custom/finetuned/checkpoint_14.pt
Saving image classifier to models/wiseft/custom/finetuned/checkpoint_14.pt
Evaluating on RN50
RN50 Top-1 accuracy: 0.8611
Results saved to results.jsonl.
Train Epoch: 14 [0% 0/21]   Loss: 1.082862  Data (t) 0.124  Batch (t) 0.846
Saving model to models/wiseft/custom/finetuned/checkpoint_15.pt
Saving image classifier to models/wiseft/custom/finetuned/checkpoint_15.pt
Evaluating on RN50
RN50 Top-1 accuracy: 0.8333
Results saved to results.jsonl.
Train Epoch: 15 [0% 0/21]   Loss: 0.864339  Data (t) 0.082  Batch (t) 1.392
Saving model to models/wiseft/custom/finetuned/checkpoint_16.pt
Saving image classifier to models/wiseft/custom/finetuned/checkpoint_16.pt
Evaluating on RN50
RN50 Top-1 accuracy: 0.8333
Results saved to results.jsonl.
Train Epoch: 16 [0% 0/21]   Loss: 0.649688  Data (t) 0.128  Batch (t) 0.876
Saving model to models/wiseft/custom/finetuned/checkpoint_17.pt
Saving image classifier to models/wiseft/custom/finetuned/checkpoint_17.pt
Evaluating on RN50
RN50 Top-1 accuracy: 0.7778
Results saved to results.jsonl.
Train Epoch: 17 [0% 0/21]   Loss: 0.560998  Data (t) 0.205  Batch (t) 1.278
Saving model to models/wiseft/custom/finetuned/checkpoint_18.pt
Saving image classifier to models/wiseft/custom/finetuned/checkpoint_18.pt
Evaluating on RN50
RN50 Top-1 accuracy: 0.7500
Results saved to results.jsonl.
Train Epoch: 18 [0% 0/21]   Loss: 0.761688  Data (t) 0.155  Batch (t) 1.089
Saving model to models/wiseft/custom/finetuned/checkpoint_19.pt
Saving image classifier to models/wiseft/custom/finetuned/checkpoint_19.pt
Evaluating on RN50
RN50 Top-1 accuracy: 0.8750
Results saved to results.jsonl.
Train Epoch: 19 [0% 0/21]   Loss: 0.621387  Data (t) 0.065  Batch (t) 0.995
Saving model to models/wiseft/custom/finetuned/checkpoint_20.pt
Saving image classifier to models/wiseft/custom/finetuned/checkpoint_20.pt
Evaluating on RN50
RN50 Top-1 accuracy: 0.8750
Results saved to results.jsonl.
Loading image classifier from models/wiseft/custom/zeroshot.pt
Loading image classifier from models/wiseft/custom/finetuned/checkpoint_20.pt
Saving image classifier to models/wiseft/custom/finetuned/wise_ft_alpha=0.000.pt
Evaluating on RN50
RN50 Top-1 accuracy: 0.0000
Results saved to results.jsonl.
Saving image classifier to models/wiseft/custom/finetuned/wise_ft_alpha=0.100.pt
Evaluating on RN50
RN50 Top-1 accuracy: 0.0000
Results saved to results.jsonl.
Saving image classifier to models/wiseft/custom/finetuned/wise_ft_alpha=0.200.pt
Evaluating on RN50
RN50 Top-1 accuracy: 0.0278
Results saved to results.jsonl.
Saving image classifier to models/wiseft/custom/finetuned/wise_ft_alpha=0.300.pt
Evaluating on RN50
RN50 Top-1 accuracy: 0.2500
Results saved to results.jsonl.
Saving image classifier to models/wiseft/custom/finetuned/wise_ft_alpha=0.400.pt
Evaluating on RN50
RN50 Top-1 accuracy: 0.4306
Results saved to results.jsonl.
Saving image classifier to models/wiseft/custom/finetuned/wise_ft_alpha=0.500.pt
Evaluating on RN50
RN50 Top-1 accuracy: 0.5139
Results saved to results.jsonl.
Saving image classifier to models/wiseft/custom/finetuned/wise_ft_alpha=0.600.pt
Evaluating on RN50
RN50 Top-1 accuracy: 0.6250
Results saved to results.jsonl.
Saving image classifier to models/wiseft/custom/finetuned/wise_ft_alpha=0.700.pt
Evaluating on RN50
RN50 Top-1 accuracy: 0.6667
Results saved to results.jsonl.
Saving image classifier to models/wiseft/custom/finetuned/wise_ft_alpha=0.800.pt
Evaluating on RN50
RN50 Top-1 accuracy: 0.7500
Results saved to results.jsonl.
Saving image classifier to models/wiseft/custom/finetuned/wise_ft_alpha=0.900.pt
Evaluating on RN50
RN50 Top-1 accuracy: 0.8472
Results saved to results.jsonl.
Saving image classifier to models/wiseft/custom/finetuned/wise_ft_alpha=1.000.pt
Evaluating on RN50
RN50 Top-1 accuracy: 0.8750
Results saved to results.jsonl.

[mcdulltii]

For anyone with a custom dataset that wishes to use the model for inference, refer to the eval_single_dataset() function in src/models/eval.py. The true labels, inference logits can be found in all_labels and all_preds (Line 31) respectively, whereas the predicted label for every image in the dataset can be found in pred (Line 50). Extract them after running the function on the custom dataset for any post visualizations/processings.