search

VinAIResearch / PhoGPT

PhoGPT: Generative Pre-training for Vietnamese (2023)
Apache License 2.0
739 stars 64 forks source link

Evaluation lack of context #25

Closed vTuanpham closed 3 months ago

vTuanpham commented 3 months ago

image

Is the question suppose to be this lack of context ? Given that only the Ground truth even mention about Vietnam.

datquocnguyen commented 3 months ago

I don't quite understand your question. Please could you elaborate a bit more?

vTuanpham commented 3 months ago

Hi @datquocnguyen . what i meant is that the question doesn't explicitly mention Vietnam, so it's reasonable to assume that it pertains to a global context rather than being specific to Vietnam. This is evidenced by the failure of all models to provide satisfactory answers to both questions.

datquocnguyen commented 3 months ago

It should be reasonable to assume that the question asks about Vietnam rather than the global context, as it's written in Vietnamese. You can think about other questions such as "vượt đèn đỏ phạt bao nhiêu tiền." If you'd like to ask about the global context, you should specify that in the input prompt.