search

CherBass / ICAM

ICAM: Interpretable Classification via Disentangled Representations and Feature Attribution Mapping
MIT License
53 stars 10 forks source link

Accuracy and F1 #8

Open waiterxiaoyy opened 11 months ago

waiterxiaoyy commented 11 months ago

Hello, I'd like to inquire about the accuracy and F1 score within the training code. Specifically, the original code uses a batch size of 2, so would the accuracy then be limited to 3 values: 0, 0.5, and 1?Also, regarding the F1 score, would it typically produce discrete values like 0.33, 0.66, and 1? Is my understanding correct?

ecr05 commented 11 months ago

Hi,

I'm not sure I can answer all these questions but given time of year, and the fact Cher has moved to industry, I'm not sure she can reply.

The accuracy would be calculated across the whole training set, how this is passed (i.e batch size) is irrelevant. Everything will get computed after everything has been passed. I don't really know/understand your question about the F1 score.

Emma

Dr Emma C. Robinson

https://metrics-lab.github.io/

Apologies if you get emails outside normal work hours - I have small children and work flexibly. I don't expect a reply at these times.

Senior Lecturer In Medical Image Computing

School of Biomedical Engineering and Imaging Sciences

Kings College London

Office: 5th floor, Becket House, 1 Lambeth Palace Road, London SE1 7EU, UK Mail: 4th Floor, Lambeth Wing, St Thomas' Hospital, London SE1 7EH Tel. +44 (0)20 784 89557

From: Zhou Yiyang @.> Sent: Friday, December 15, 2023 11:54 AM To: CherBass/ICAM @.> Cc: Subscribed @.***> Subject: [CherBass/ICAM] Accuracy and F1 (Issue #8)

Hello, I'd like to inquire about the accuracy and F1 score within the training code. Specifically, the original code uses a batch size of 2, so would the accuracy then be limited to 3 values: 0, 0.5, and 1?Also, regarding the F1 score, would it typically produce discrete values like 0.33, 0.66, and 1? Is my understanding correct?

— Reply to this email directly, view it on GitHubhttps://github.com/CherBass/ICAM/issues/8, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AD7GXPFC3BL3L4RPFKZGIVLYJQ3BFAVCNFSM6AAAAABAWKVFQ2VHI2DSMVQWIX3LMV43ASLTON2WKOZSGA2DGNJZGU4TEMY. You are receiving this because you are subscribed to this thread.Message ID: @.***>

CherBass commented 11 months ago

Hi,

Thanks for your question. So as Emma mentioned the way metrics are computed is that first predictions are made for the entire dataset, and then metrics are computed, so the batch size is irrelevant in the inference stage. F1 can be a range of 0-1 not discrete.