Closed anicolson closed 1 year ago
Hello,
Yes the triple is respectively (RG_E, RGER, RG\hat ER) This also corresponds to simple, partial, complete.
I noticed you use the keyword "full" (to get all the rewards), please have a look at the updated code: https://github.com/jbdel/vilmedic/blob/main/vilmedic/blocks/scorers/scores.py#L99
RadGraph is now a pip package and uses the term "all" to get all the reward. You can use the package as such:
pip install radgraph
from radgraph import F1RadGraph
refs = ["no acute cardiopulmonary abnormality",
"no evidence of acute pulmonary process moderately large size hiatal hernia"]
hyps = ["no acute cardiopulmonary abnormality",
"no evidence of acute cardiopulmonary process moderate hiatal hernia"]
f1radgraph = F1RadGraph(reward_level="partial")
score, _, hypothesis_annotation_lists, reference_annotation_lists = f1radgraph(hyps=hyps,
refs=refs)
print(score)
#0.7142857142857143
You can also use F1Chexbert that way, for more info see: https://vilmedic.app/misc/bionlp23/sharedtask
Also, you are very much welcome to participate to the challenge :-)
Thanks.
JB
Thank you JB, great stuff with the radgraph package!
Btw, that last link did not work.
Aaron.
you mean this : https://vilmedic.app/misc/bionlp23/sharedtask You should find a page with section 3.2 F1CheXbert detailing the package usage.
JB
Hi JB,
That link is not working for me sorry,
Aaron
I'm sorry to hear that. I'm not sure whats the issue is. everything seems okay worldwide https://downforeveryoneorjustme.com/vilmedic.app?proto=https what do you see? is it a security issue?
Hi JB,
Its working now, no idea why it wasn't before. Very cool, I am definetly interested in challenge 2.2.
Thanks, Aaron.
Hi,
When initialising RadGraph with reward_level='full', the first element of the returned tuple has three mean scores, which I assume are the three RadGraph rewards/metrics from your paper:
I was just wondering if you could let me know the order of these in the returned value when reward_level='full'? And which is used with reward_level='partial'? Sorry, I may of missed this in your code.
Thank you! Aaron.