Support for creating prompt-reply pair tasks

[iurimatias]

Motivation

Currently a user can contribute an initial prompt but can't reply to it. Instead someone else is expected to reply to that prompt as the Assistant. This can be challenging as prompts often require specialized knowledge and/or the reply was expected to be something specific the prompter had in mind that might not be obvious. Currently the majority of the pending tasks are "Reply as Assistant", accounting for 60-80% of all pending tasks, so this is a sort of bottleneck.

Typically when a user contributes an initial prompt they already have a pretty good idea of what sort of reply they expect and are in the better position to do that reply. It might be a prompt in a topic they are passionate about or have expert knowledge. For example someone who loves chess could contribute a specific prompt such as "What are the typical plans for white in the Scheveningen?" and know what a good prompt would be. On the otherhand a user that doesn't know about chess in depth and without any clues will have no idea what to reply and might not even know the question is about chess. And if they do try to reply it might end up with a low quality reply like "Scheveningen is in the Netherlands. I'm not sure what you mean about plans for white."

Proposal

Add support for a "Prompt & Reply" task that a user can specify both the initial prompt and the assistant reply. This can go through the spam mechanism, and appear again as a "Reply as Assistant" task so other users can also reply to that prompt. Perhaps with the original reply shown as an example as well so they can also improve upon it. Alternatively, this could also be done without adding a new task but instead by adding a checkbox to "Create Initial Prompts" task to optionally add a reply as well.

Backend wise, a possible option is to add directly to the tree both the prompt and the reply attached to it, and create tasks to classify that prompt and further tasks to reply to the original prompt.

[shuyi56]

Hi, I would like to try implementing this feature. I like the alternative proposal of a checkbox in the initial prompt task. Perhaps something like this?

[notmd]

Copied from #1882 Solution for #1030.

• Option 1 (Andreas suggestion): Add the ability to create a reply for a prompt. After that, the message tree will go to prompt_lottery_waiting state.
  • Easy to implement and fit with our design, but the problem here is we have so many prompt_lottery_waiting trees, which can take months to complete. But in near future, most of our assistant replies will be synthetic then the waiting prompts maybe decrease dramatically.
  • Option 2: As Huu said a single Q/A message tree is fine (https://discord.com/channels/1055935572465700980/1059076134807023656/1078649949119598592). Add the ability to create a reply for a prompt. After that, the tree will go to promtp_reply_pair_review state. And after receiving N good review it will be ready_for_export otherwise will be halted_by_moderator. Because the tree only has a single Q/A we don't need to rank it. We also don't want it to grow more because someone may flood our task with this and we will need a lottery for it again. We also need to extract the review task of the prompt reply pair to a separate task
  • Good for simple Q/A and can increase our dataset a lot.
  • Much more complex implementation, more tasks.
  • Option 3: Just like option 1 but we only give permission to legit/verified users (maybe users in the leaderboard or something). The tree will be growing immediately after creation. Also, we can limit X prompt reply pair that can be created per day (globally or per user).

[KosmasKanellakis]

Hi, me and @IoannaNtousaki would like to be assigned and work on the project.

[IoannaNtousaki]

Hello. @kosmass2001 and I are working together for an assignment. Is there any chance we can be both assigned in the same issue?

[notmd]

Hi @kosmass2001 @IoannaNtousaki, thank you for your interest in helping us. Can you write a short summary of your idea/solution?

[IoannaNtousaki]

Hi @kosmass2001 @IoannaNtousaki, thank you for your interest in helping us. Can you write a short summary of your idea/solution?

Hello again. We were thinking of multiple different solutions on this issue. We still examine the possibility of adding the checkbox functionality, but there is also an idea of expanding the utility of the feature by creating an additional frame in the page that will give the user the chance to submit his answer automatically. Even more we could use some kind of redirection to the "answer" page after the question submission, displaying the question that was just created and allowing the user to answer it himself. As for the backend, we were thinking of implementing the third suggestion combining "prompt_lottery_waiting" and "user verification". Although, we need to examine the code a bit more so that we can have a more conclusive perspective of the potential changes framework. We want to create a feature that will be both convenient for the user and practical out of a programming perspective.

I'd like to mention that our contribution in the project is a part of a course assignment and we are getting graded upon the quality of the code and its high implementation standards, so we will try to figure out the best solution.

[Michal-Mikolas]

I was just going to create the same feature-request and found this one.

And I 100 % agree with the author. I believe that having possibility to create prompt AND the answer will boost both quality and quantity of the data created by volunteers.

The reason is that I (and I believe other people as well) am solving so many problems in my life. These are often problems connected to my expert profession. And I often spend one or more hours solving them. So after that I am sure I can create one or more very good quality question/answer pairs backed up by a few hours of my time spent on researching the problem 👍

Also, after I create the question & answer in english, I can create the same good quality content in my native language (which is not English). A lot of volunteers here are not native English speakers, so they will probably do the same.

---

All of this will finally lead to:

• Better quality in both prompts and answers.
• More data handwritten by volunteers (no ChatGPT copy-paste).
• These high quality data will also populate more languages other than English.

[andreaskoepf]

We are looking for a developer to work on this highly requested issue.

[Michal-Mikolas]

I tried, but as PHP developer, I've got completely lost in that React code 😬