RadGraph

[anicolson]

Hi,

When initialising RadGraph with reward_level='full', the first element of the returned tuple has three mean scores, which I assume are the three RadGraph rewards/metrics from your paper: I was just wondering if you could let me know the order of these in the returned value when reward_level='full'? And which is used with reward_level='partial'? Sorry, I may of missed this in your code.

Thank you! Aaron.

[jbdel]

Hello,

Yes the triple is respectively (RG_E, RGER, RG\hat ER) This also corresponds to simple, partial, complete.

I noticed you use the keyword "full" (to get all the rewards), please have a look at the updated code: https://github.com/jbdel/vilmedic/blob/main/vilmedic/blocks/scorers/scores.py#L99

RadGraph is now a pip package and uses the term "all" to get all the reward. You can use the package as such:

pip install radgraph

from radgraph import F1RadGraph
refs = ["no acute cardiopulmonary abnormality",
        "no evidence of acute pulmonary process moderately large size hiatal hernia"]

hyps = ["no acute cardiopulmonary abnormality",
        "no evidence of acute cardiopulmonary process moderate hiatal hernia"]

f1radgraph = F1RadGraph(reward_level="partial")
score, _, hypothesis_annotation_lists, reference_annotation_lists = f1radgraph(hyps=hyps,
                                                                               refs=refs)
print(score)
#0.7142857142857143

You can also use F1Chexbert that way, for more info see: https://vilmedic.app/misc/bionlp23/sharedtask

Also, you are very much welcome to participate to the challenge :-)

Thanks.

JB

[anicolson]

Thank you JB, great stuff with the radgraph package!

Btw, that last link did not work.

Aaron.

[jbdel]

you mean this : https://vilmedic.app/misc/bionlp23/sharedtask You should find a page with section 3.2 F1CheXbert detailing the package usage.

JB

[anicolson]

Hi JB,

That link is not working for me sorry,

Aaron

[jbdel]

I'm sorry to hear that. I'm not sure whats the issue is. everything seems okay worldwide https://downforeveryoneorjustme.com/vilmedic.app?proto=https what do you see? is it a security issue?

[anicolson]

Hi JB,

Its working now, no idea why it wasn't before. Very cool, I am definetly interested in challenge 2.2.

Thanks, Aaron.