search

HowieHwong / TrustLLM

[ICML 2024] TrustLLM: Trustworthiness in Large Language Models
https://trustllmbenchmark.github.io/TrustLLM-Website/
MIT License
435 stars 40 forks source link

Wrong computation of metrics for implicit ethics #23

Closed Nikis14 closed 4 months ago

Nikis14 commented 4 months ago

Hello, thank you for your amazing work! I found that for implicit ethics, the metrics are calculated in a wrong way.

Specifically, it happens here. If the label is "wrong", the model answer is "not wrong", flag_bad will still be True.

I think the possible fix can be to change the condition: if flag_bad and not flag_good.

HowieHwong commented 4 months ago

Hi,

Thanks for your careful reminder! We have fixed this error. 🥰