geekan / MetaGPT

🌟 The Multi-Agent Framework: First AI Software Company, Towards Natural Language Programming
https://deepwisdom.ai/
MIT License
45.54k stars 5.42k forks source link

[Very important] metagpt need to send some question and related choices to human, to get feedback from human #1536

Open CoderYiFei opened 1 month ago

CoderYiFei commented 1 month ago

It needs to do:

  1. During its developing (1) Is this part a key point during total developing circle? If so, (or human decide to) , trigger (2)... (2) What kind of question and choices should be sent to human? (3) Then send question and choices to human and receive feedback (4) How to change its following work, after receiving human choices.

  2. At the begging of work And at the begging of incremental work(or normal work), It also needs to show its understanding of current user's code repo and user's requirement.....and to let human to answer "correct understanding" or "incorrect understanding" additionally, sending key question and choices it designed , to clarify its understanding.

Product rules: To choose "A B C D", or to answer "correct or incorrect" , is easy for human. To write complex words and code, is hard for human.

geekan commented 1 month ago

Yes. These are available in our 1.0 version. However, this problem is actually not as simple as we think. For example, "Buy me an apple" is simple for humans, but it is very complex for machines.

Because we don’t know whether we are looking for mobile phones or fruits, we don’t know the required brand, quality, price, quantity, distance, time, transaction person, transaction channel, transaction method, etc. "Buy three apples that ripen on a full moon night on the Tianshan Mountains, buy them within a month, trade with Bitcoin, pay later" also meets the requirements of "buying apples".

Although we would intuitively say that it has some optimization points, it is very difficult to completely solve this problem.

CoderYiFei commented 1 month ago

Yes. These are available in our 1.0 version. However, this problem is actually not as simple as we think. For example, "Buy me an apple" is simple for humans, but it is very complex for machines.

Because we don’t know whether we are looking for mobile phones or fruits, we don’t know the required brand, quality, price, quantity, distance, time, transaction person, transaction channel, transaction method, etc. "Buy three apples that ripen on a full moon night on the Tianshan Mountains, buy them within a month, trade with Bitcoin, pay later" also meets the requirements of "buying apples".

Although we would intuitively say that it has some optimization points, it is very difficult to completely solve this problem.

At the beginning, building accurate user 's portrait and task's doc by llm itself , is pretty important... It can narrow down the following project 's question range.

LimFang commented 1 month ago

It needs to do:

  1. During its developing (1) Is this part a key point during total developing circle? If so, (or human decide to) , trigger (2)... (2) What kind of question and choices should be sent to human? (3) Then send question and choices to human and receive feedback (4) How to change its following work, after receiving human choices.
  2. At the begging of work And at the begging of incremental work(or normal work), It also needs to show its understanding of current user's code repo and user's requirement.....and to let human to answer "correct understanding" or "incorrect understanding" additionally, sending key question and choices it designed , to clarify its understanding.

Product rules: To choose "A B C D", or to answer "correct or incorrect" , is easy for human. To write complex words and code, is hard for human.

myabe somthing like autogen? now the agent message subscription mechanism is not enough though

CoderYiFei commented 1 month ago

You mean metagpt's agent message subscription mechanism not good enough, or you mean general mechanism and theory, not good enough in all alternatives projects

LimFang commented 1 month ago

Message subscription in metagpt is more likely a centralized pool pattern. Every agent pulls and pushes its designed or desired information. That is not good enough for timely interaction between machines and humans.

geekan commented 1 month ago

My design intent is to be a pure pubsub, and can be replaced by a queue. I understand pubsub solves all problems.