Together-Java / TJ-Bot

TJ-Bot is a Discord Bot used on the Together Java server. It is maintained by the community, anyone can contribute.
https://togetherjava.org
GNU General Public License v3.0
100 stars 83 forks source link

Title generation by ChatGPT for transfer-questions command #1059

Closed vishv843 closed 2 months ago

vishv843 commented 4 months ago

Previously, the title was simply created by putting the message when /transfer-questions command was invoked. The title had to be written by a human for clarity. This PR solves that issue by using ChatGPT for title generation. #1011

question response

vishv843 commented 4 months ago

Hi @ankitsmt211. I have made the necessary changes. Please review it.

vishv843 commented 4 months ago

I tested the prompt you gave on some questions, worked pretty well. I have added it. Thanks!!

ankitsmt211 commented 4 months ago

@vishv843 everything looks great, i wish i should have tested the prompt more before suggesting it. I tested it on a couple of more questions and it would return title in quotation marks sometimes, so i finally decided to ask gpt itself for a better prompt. Now this one i validated against more than 4-5 questions from server itself so seems to be solid overall.

image

String chatGptPrompt = "Create a concise title for the question: '%s'. The title should be 4-5 words maximum and presented without quotation marks."
    .formatted(originalMessage);

I'd be fine with doing the changes myself if it's a bother 😅

Taz03 commented 4 months ago

@ankitsmt211 imo its better to just check for quotes and remove them manually, gpt sometimes does things even when told not to

if (title.matches("['\"]\\w+['\"]")) title = title.substring(1, gptTitle.length() - 1);
ankitsmt211 commented 4 months ago

@ankitsmt211 imo its better to just check for quotes and remove them manually, gpt sometimes does things even when told not to

if (title.matches("['\"]\\w+['\"]")) title = title.substring(1, gptTitle.length() - 1);

this is very specific and if GPT is unreliable then we can't really handle all other cases, so imo just trying for best prompt that we can should be a priority, if you're specific with prompt it does work as expected specially when generating simple text is what it's known about.