Missing weighted_loss()

[harshaUwm163]

In line 34 of risk_histogram.py in the imagenet directory, there is a reference to the weighted_loss() which is not defined.

Can you tell me where to look for it or can you update the repo with the definition?

Thanks!

[aangelopoulos]

Huh, that's so weird. Thanks for pointing it out. I'll go look for it.

[currywhite30]

Huh, that's so weird. Thanks for pointing it out. I'll go look for it.

Has this issue been resolved? Thanks

[aangelopoulos]

Fixed -- let me know if it works for you.

[currywhite30]

Fixed -- let me know if it works for you.

Thank you very much for your timely reply. The code can run, but an error occurs when drawing the distribution histogram. What does this weighted_loss function mean? Weighted_loss is not 0 when both est_labels and labels are 1. On the contrary, weighted_loss is 0 when the predicted category is different from the real label. This seems to be contrary to my understanding. Can you explain this? Sorry to bother you!

[aangelopoulos]

You're right. Can you set

def weighted_loss(est_labels, labels, losses):
    return (est_labels * (1-labels) * losses[None,:]).sum(dim=1)

And let me know if it works for you? If so, I'll push the update.

[currywhite30]

You're right. Can you set

def weighted_loss(est_labels, labels, losses):
    return (est_labels * (1-labels) * losses[None,:]).sum(dim=1)

And let me know if it works for you? If so, I'll push the update.

Sorry, it doesn't work. And I try to define the weightedloss function as follws, which is consistent with $L(y, \mathcal{S})=L{y} \mathbb{I}_{{y \notin \mathcal{S}}}$

def weighted_loss(est_labels, labels, losses):
    current_labels = np.argmax(labels, axis=1)
    mask = (est_labels[np.arange(len(labels)), current_labels] == 0)
    weighted_loss = np.where(mask, losses[current_labels], 0)
    return weighted_loss

Then the distribution histogram of risk seems to be correct. But the distribution histogram of size appears to be concentrated on 1 and 2, which is not consistent with the example given in the repository. Is it caused by model? My result is shown below:

[aangelopoulos]

Hey, thanks so much for fixing that. Want to make a PR, which I can merge? Otherwise I'll just edit myself.

Yeah, it looks like the model you're using must have a higher accuracy than the one I used. I know there has since been an upgrade to the models in the torchvision repo, which might be the root cause.

In any case, this is totally reasonable/expected behavior :)

[currywhite30]

Hey, thanks so much for fixing that. Want to make a PR, which I can merge? Otherwise I'll just edit myself.

Yeah, it looks like the model you're using must have a higher accuracy than the one I used. I know there has since been an upgrade to the models in the torchvision repo, which might be the root cause.

In any case, this is totally reasonable/expected behavior :)

Thanks! I have made a PR, please take a look.