search

ijyliu / anlp23-project

An empirical study of the costs and practicalities of prompt engineering techniques on standard and novel benchmarks
0 stars 0 forks source link

Does GPT-4 "think" in other languages? #23

Closed ijyliu closed 11 months ago

ijyliu commented 11 months ago

Is it translating under the hood?

Does splitting tasks in foreign languages into translation then task then translation improve performance?

ijyliu commented 11 months ago

This is very hard for me to evaluate, as it requires being able to judge outputs in a foreign language.

What I am proposing is similar to "pivot prompting" (https://arxiv.org/pdf/2301.08745.pdf), except there is a task performed in the middle.

ijyliu commented 11 months ago

Possibly return to this, but only on the condition of having another group member that is good at evaluating foreign language tasks.

Or finding some way to get automated evaluation... but that seems dicey.