tips for retraining/transfer training

[tripzero]

Trying to transfer train with pure pytorch but can't seem to get good results. Often the output of the model produces just two tags (or no tags) after being trained. My current loss function and optimizer are:

criterion = nn.BCEWithLogitsLoss()
optimizer_ft = optim.SGD(model.parameters(), lr=0.001, momentum=0.9)
scheduler = lr_scheduler.StepLR(optimizer_ft,
                                    step_size=7,
                                    gamma=0.1)

I'm not very familiar with fastai, so I can't tell which loss function or optimizer you ended up using. Any help would be appreciated.

[RF5]

Hmm, its hard to know the details of how you are trying to perform transfer learning. The original training in fastai used the same BCEWithLogitsLoss (but with Fastai's flattened version here).

Are you sure you are using the same normalization as with the original model, i.e are you sure you have the transform

transforms.Normalize(mean=[0.7137, 0.6628, 0.6519], std=[0.2970, 0.3017, 0.2979]),

somewhere applied to your input before you feed it into the model? There are a few other things which might be causing problems, you might want to try other optimizers such as Adam (what was used during training). The learning rate might also be a bit high if it starting to output very strange things for all inputs. Best of luck!