search

deeplearning-wisc / gradnorm_ood

On the Importance of Gradients for Detecting Distributional Shifts in the Wild
Apache License 2.0
53 stars 7 forks source link

Question about gradients for ood #12

Open ReusJeffery opened 11 months ago

ReusJeffery commented 11 months ago

Excellent work,But I'm very confused about a question.For example, there are a batch of about 2,000 samples. The samples contain both in-distribution data and out-of-distribution data. How can I identify these out-of-distribution data based on gradients through your work?

YixuanLi commented 11 months ago

Thanks for your interest in our work. For testing, you need to rely on a threshold to decide whether each sample in a batch is ID or OOD. The threshold is chosen based on the ID data (e.g., at 95% percentile).

On Thu, Nov 30, 2023 at 5:58 AM ReusJeffery @.***> wrote:

Excellent work,But I'm very confused about a question.For example, there are a batch of about 2,000 samples. The samples contain both in-distribution data and out-of-distribution data. How can I identify these out-of-distribution data based on gradients through your work?

— Reply to this email directly, view it on GitHub https://github.com/deeplearning-wisc/gradnorm_ood/issues/12, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABUGVB262BLTLZXWQWC63NDYHBYGDAVCNFSM6AAAAABABB3ZA2VHI2DSMVQWIX3LMV43ASLTON2WKOZSGAYTQNBZHAZTMNQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

ReusJeffery commented 11 months ago

Thanks for your interest in our work. For testing, you need to rely on a threshold to decide whether each sample in a batch is ID or OOD. The threshold is chosen based on the ID data (e.g., at 95% percentile). … On Thu, Nov 30, 2023 at 5:58 AM ReusJeffery @.> wrote: Excellent work,But I'm very confused about a question.For example, there are a batch of about 2,000 samples. The samples contain both in-distribution data and out-of-distribution data. How can I identify these out-of-distribution data based on gradients through your work? — Reply to this email directly, view it on GitHub <#12>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABUGVB262BLTLZXWQWC63NDYHBYGDAVCNFSM6AAAAABABB3ZA2VHI2DSMVQWIX3LMV43ASLTON2WKOZSGAYTQNBZHAZTMNQ . You are receiving this because you are subscribed to this thread.Message ID: @.>

Thank you for your response. Can I understand it in this way: assuming that the threshold is set at 95%, according to the method described in your paper, if we calculate the gradients and sort them in ascending order, then the top 5% would be identified as OOD (out-of-distribution)?