Hi, I am trying to run your code but I'm getting an error relating to the DataLoader.
I have a pre-trained model, model1.pth, and a DataLoader object data.valid_dl.dl. This is the validation data I used for training.
The DataLoader seems correct to me. Here are some checks:
I'm trying to run this within fastAI, here's what I was trying to do.
from temperature_scaling import ModelWithTemperature
# Load data
data = ImageDataBunch.from_folder(path)
# Train
learn = cnn_learner(data, models.resnet34, metrics=accuracy)
learn.fit_one_cycle(10)
learn.save('model1')
orig_model = ".../model1.pth" # create an uncalibrated model somehow
valid_loader = data.valid_dl.dl # Create a DataLoader from the SAME VALIDATION SET used to train orig_model
scaled_model = ModelWithTemperature(orig_model)
scaled_model.set_temperature(valid_loader)
I get the following error:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-12-d4da6fd75710> in <module>()
6
7 scaled_model = ModelWithTemperature(orig_model)
----> 8 scaled_model.set_temperature(valid_loader)
/content/temperature_scaling/temperature_scaling.py in set_temperature(self, valid_loader)
46 for input, label in valid_loader:
47 input = input.cuda()
---> 48 logits = self.model(input)
49 logits_list.append(logits)
50 labels_list.append(label)
TypeError: 'str' object is not callable
I'm probably misunderstanding what valid_loader has to contain. Any help would be more than welcome.
I've also started a thread about calibration in the fastai forums, citing your paper, talk and code, so if you would rather reply there to reach a larger audience, that'd be great. Here's a link to the thread:
Hi, I am trying to run your code but I'm getting an error relating to the DataLoader. I have a pre-trained model,
model1.pth
, and a DataLoader objectdata.valid_dl.dl
. This is the validation data I used for training.The DataLoader seems correct to me. Here are some checks:
I'm trying to run this within fastAI, here's what I was trying to do.
I get the following error:
I'm probably misunderstanding what
valid_loader
has to contain. Any help would be more than welcome.I've also started a thread about calibration in the fastai forums, citing your paper, talk and code, so if you would rather reply there to reach a larger audience, that'd be great. Here's a link to the thread:
https://forums.fast.ai/t/calibrating-your-network-to-get-better-probability-estimates/41425
Thank you!