gnehs / subtitle-translator

Translate subtitle using ChatGPT
MIT License
344 stars 46 forks source link

System instruction & JSON structure result in unstable response #9

Open Francoyy opened 1 year ago

Francoyy commented 1 year ago

Seems that the instruction given to ChatGPT is no longer working correctly. I would be curious to know where the instruction

The JSON format returned by ChatGPT is not always the same. completion.data.choices[0].message.content sometimes contains another key "translate" inside, sometimes not. It makes the parsing of the response unstable, with "Undefined" showing up sometimes. Sometimes, it is the source language that displays, and not the target language. But after checking the actual response from ChatGPT, the translation is given, but the script is not fetching the translation correctly.

This makes me wonder 2 things, if the author could share a little background on this: 1) Where the instruction comes from and why: You are a program responsible for translating subtitles. Your task is to output the specified target language based on the input text. Please do not create the following subtitles on your own. Please do not output any text other than the translation. You will receive the subtitles as array that needs to be translated, as well as the previous translation results and next subtitle. If you need to merge the subtitles with the following line, simply repeat the translation. Please transliterate the person's name into the local language. Target language: ${config.TARGET_LANGUAGE}

2) Why having a JSON structure containing Input, Next, and keeping the previous request into previousSubtitles and giving to ChatGPT as well.

Since the instruction in 1) does not explain the JSON structure of Input / Next, I feel like in many cases, ChatGPT does not correctly understand that JSON structure. In some cases, it also translates what is given in Next.

Francoyy commented 1 year ago

I feel like there is a lot of thought given in the structure and it's supposed to give ChatGPT more context for better translations, but somehow the structure / instructions make the response unstable. Would the author suggest some try&error on the instruction and JSON structure?