search

wxr99 / HolisticPU

Beyond Myopia: Learning from Positive and Unlabeled Data through Holistic Predictive Trends [NeurIPS 2023]
9 stars 2 forks source link

F1 score #1

Closed chmxu closed 11 months ago

chmxu commented 11 months ago

Thank you for this inspiring work! I have a question about the calculation of F1 score. As I can understand the data loading code snippet, the positive samples have label 0 (https://github.com/wxr99/HolisticPU/blob/main/dataset/cifar.py#L170). However when calculating F1 score in (https://github.com/wxr99/HolisticPU/blob/main/utils/misc.py#L46) the sklearn function f1_score used set default pos_label=1. I am not sure if this would lead to some problems.

wxr99 commented 11 months ago

thx for your question, we will mention or revise it later. It generally doesn't affect the results as long as the given y_true and y_pred are matched. Since F1 = 2 (precision recall) / (precision + recall), switching the notation of positive and negative class is equivalent to switching precision and recall which will not affect the result.

wxr99 commented 11 months ago

But we admit this may provoke some confusion, so we will revise it after the issue is closed, thx again.

chmxu commented 11 months ago

Thank you for your reply!