Questions about gpt_interviewer 关于gpt_interviewer的一些疑问

[GolfHotelSierra]

Problem Description 问题描述

Thank you very much for your work. After reading the paper and code, I have some questions regarding the gpt_interviewer part and I hope to get clarification.

非常感谢你们的工作。在阅读了论文和代码后我对 gpt_interviewer 这部分的内容有一些疑问，希望可以得到解答

Question1 疑问1：

In run_api_interview_turns.py, on line 123 and line 59, it seems that the value corresponding to the question key in generated_agent_interview_for_multiturn_{name}.json is being passed as the topic. Why is the existing topic key from generated_agent_interview_for_multiturn_{name}.json not being used?

在 run_api_interview_turns.py 中的第 123 行，和第 59 行中，看起来是将 generated_agent_interview_for_multiturn_{name}.json 中的 question 键对应的值作为 topic 传入了。请问为什么没有使用 generated_agent_interview_for_multiturn_{name}.json 中本来就存在的 topic 键呢？

# line123
task_list.append({'qid': idx, 'question': q['question'], 'character_name': name, 'model_type': model_type, 'output_path': output_path})

# line59
topic = item['question']
interviewer.set_topic_and_profile(topic, character_profile)

This is where the placeholder {topic} from the default prompt in agent_meta_prompt_interviewer_chatgpt.txt is filled in,

也就是代码中默认填入的提示词 agent_meta_prompt_interviewer_chatgpt.txt 里的 {topic} 占位符，

I want you to act as an curious man who has interested at {character}. And I will act as the character and you will chat with me. I want you to only reply as a curious person. Your task is to elicit the memory, values and personality of the character as detailed as possible. If {character} dodge the questions by saying things without details, you can ask follow-up questions. Do not get off the topic. Do not mention the name of the character. Just use "you" to refer the character. Do not write all the conservation at once. Do not write explanations. Ask me the questions one by one and wait for my response. Below is some context about this meeting. You can ask me previous questions again to see if I am consistent to the answer.

The goal of this conversation is:
{topic}

The profile of the character:
{profile}

The status of us is as follows:
Location: {loc_time}
Status: {status}

Example output:
Character1 (speaking): Detailed utterance ...

Character2 (speaking): Detailed utterance ...

The conversation begins:

For example, in data/seed_data/questions/generated_agent_interview_for_multiturn_Beethoven.json, the topic key is,

比如，data/seed_data/questions/generated_agent_interview_for_multiturn_Beethoven.json 中的 topic 键，

{
    "topic_id": "topic-3",
    "topic": "values and preferences",
    "question": "Ludwig, what preference for role do you attribute to your individual development during your heroic period?"
},

[choosewhatulike]

agent_meta_prompt_interviewer_chatgpt.txt这个prompt的作用是让GPT作为interviewer来对另一个模型（比如character-llm）进行问询。

The goal of this conversation is: {topic}

这里的topic实际上就是问询时指定的主题，对应json问题集中的question，是与角色相关的一个问题。问题集中的topic只是表示question所在的分类，不会在interview中用到，可以忽略。

[GolfHotelSierra]

感谢您的回答，不过我在尝试单独测试 gpt_interviewer 时遇到了一些问题，

根据 eval_utils.py 中293行的 get_prompt() 方法，我在 agent_meta_prompt_interviewer_chatgpt.txt 提示词的最后人工补充了一些对话，来假设采访已经进行了一定的轮次

我预期的结果是谈话会继续下去，不过实际上gpt作为采访者仍然在重复类似于 topic 的问题，

# 第一轮采访生成的问题
topic = "Ludwig, what would you say has been the greatest influence on your musical style during your early period?"
Man (speaking): What would you say has been the greatest influence on your musical style during your early period?

# 第二轮采访生成的问题
Man (speaking): What would you say has been the greatest influence on your musical style during your early period?
Ludwig van Beethoven (speaking): The influence of the times and my personal experiences are the biggest influences on my musical style.
Man (speaking): During your early period, what do you think has had the most significant impact on shaping your musical style? # 或者是其他类似的句子，仍然关注于询问对于音乐风格的影响，而没有继续采访下去

我使用的提示词和填充的占位符是这些，

I want you to act as an curious man who has interested at {character}. And I will act as the character and you will chat with me. I want you to only reply as a curious person. Your task is to elicit the memory, values and personality of the character as detailed as possible. If {character} dodge the questions by saying things without details, you can ask follow-up questions. Do not get off the topic. Do not mention the name of the character. Just use "you" to refer the character. Do not write all the conservation at once. Do not write explanations. Ask me the questions one by one and wait for my response. Below is some context about this meeting. You can ask me previous questions again to see if I am consistent to the answer.

The goal of this conversation is:
{topic}

The profile of the character:
{profile}

The status of us is as follows:
Location: {loc_time}
Status: {status}

Example output:
Character1 (speaking): Detailed utterance ...

Character2 (speaking): Detailed utterance ...

The conversation begins:

Man (speaking): What would you say has been the greatest influence on your musical style during your early period?
Ludwig van Beethoven (speaking): The influence of the times and my personal experiences are the biggest influences on my musical style.
Man (speaking):

character = "Ludwig van Beethoven"
topic = "Ludwig, what would you say has been the greatest influence on your musical style during your early period?"
status = 'Ludwig van Beethoven is casually chatting with a man from the 21st century. Ludwig van Beethoven fully trusts the man who engage in conversation and shares everything Ludwig van Beethoven knows without reservation.'
loc_time = 'Coffee Shop - Afternoon'
profile = '<使用profile目录下的人物介绍>'

[GolfHotelSierra]

此外，请问 data/seed_data/questions 中针对不同人物的这些问题，是如何收集到的？是否也可以通过 AI 生成得到？

[GolfHotelSierra]

agent_meta_prompt_interviewer_chatgpt.txt这个prompt的作用是让GPT作为interviewer来对另一个模型（比如character-llm）进行问询。

The goal of this conversation is: {topic}

这里的topic实际上就是问询时指定的主题，对应json问题集中的question，是与角色相关的一个问题。问题集中的topic只是表示question所在的分类，不会在interview中用到，可以忽略。

我使用下面的代码进行的测试，生成了多次结果都没有继续当前的采访，

from langchain.chains.llm import LLMChain
from langchain_core.prompts import PromptTemplate
from langchain_openai import ChatOpenAI

if __name__ == "__main__":
    GENERATE_MULTI_ROUND_INTERVIEW_PROMPT_TEMPLATE = """I want you to act as an curious man who has interested at {character}. And I will act as the character and you will chat with me. I want you to only reply as a curious person. Your task is to elicit the memory, values and personality of the character as detailed as possible. If {character} dodge the questions by saying things without details, you can ask follow-up questions. Do not get off the topic. Do not mention the name of the character. Just use "you" to refer the character. Do not write all the conservation at once. Do not write explanations. Ask me the questions one by one and wait for my response. Below is some context about this meeting. You can ask me previous questions again to see if I am consistent to the answer.

The goal of this conversation is:
{topic}

The profile of the character:
{profile}

The status of us is as follows:
Location: {loc_time}
Status: {status}

Example output:
Character1 (speaking): Detailed utterance ...

Character2 (speaking): Detailed utterance ...

The conversation begins:

Man (speaking): What would you say has been the greatest influence on your musical style during your early period?
Ludwig van Beethoven (speaking): The influence of the times and my personal experiences are the biggest influences on my musical style.
Man (speaking):"""

    character = "Ludwig van Beethoven"
    topic = "Ludwig, what would you say has been the greatest influence on your musical style during your early period?"
    status = 'Ludwig van Beethoven is casually chatting with a man from the 21st century. Ludwig van Beethoven fully trusts the man who engage in conversation and shares everything Ludwig van Beethoven knows without reservation.'
    loc_time = 'Coffee Shop - Afternoon'
    profile = '''<使用profile目录下的人物介绍>'''
    model_name = "gpt-3.5-turbo"
    openai_api_key = "OPENAI_API_KEY"
    openai_api_base = "OPENAI_API_BASE"

    llm = ChatOpenAI(model_name=model_name,
                     openai_api_key=openai_api_key,
                     openai_api_base=openai_api_base)
    chain = LLMChain(llm=llm,
                     prompt=PromptTemplate.from_template(template=GENERATE_MULTI_ROUND_INTERVIEW_PROMPT_TEMPLATE))
    for _ in range(3):
        result = chain.invoke(
            input={'character': character, 'topic': topic, 'profile': profile, 'loc_time': loc_time, 'status': status})[
            'text']
        print(result)

输出结果：

Can you elaborate on how your personal experiences have influenced your musical style during your early period?
Can you elaborate on how the times and your personal experiences influenced your musical style during your early period?
Can you elaborate on how the times and your personal experiences influenced your musical style during your early period?

[choosewhatulike]

生成结果受模型版本和生成参数的影响还蛮大的，可以再试试。针对不同的问题，模型一些情况下会选择追问，追问之后再提出新的问题。

[GolfHotelSierra]

生成结果受模型版本和生成参数的影响还蛮大的，可以再试试。针对不同的问题，模型一些情况下会选择追问，追问之后再提出新的问题。

好的，非常感谢 还想向您询问一下 data/seed_data/questions 中针对不同人物的这些问题，是如何收集到的？是否也可以通过 AI 生成得到？

[choosewhatulike]

对，这些问题是通过gpt-4生成后人工整理的

[GolfHotelSierra]

对，这些问题是通过gpt-4生成后人工整理的

好的，感谢您的解答