When using the parameter extractor in the parameter extraction node, an error will be reported when selecting the Claude3 series model and the Gemini series model

Self Checks

[X] This is only for bug report, if you would like to ask a question, please head to Discussions.
[X] I have searched for existing issues search for existing issues, including closed ones.
[X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
[X] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
[X] Please do not modify this template :) and fill in all the required fields.

Dify version

0.6.12-fix1

Cloud or Self Hosted

Self Hosted (Source)

Steps to reproduce

When using the parameter extractor in the parameter extraction node, an error will be reported when selecting the Claude3 series model and the Gemini series model Error content: [vertex_ai] Bad Request Error, 400 Please ensure that multiturn requests alternate between user and model.

✔️ Expected Behavior

Normal reasoning and return

❌ Actual Behavior

[vertex_ai] Bad Request Error, 400 Please ensure that multiturn requests alternate between user and model.

@wangiii Hello there! I'm here to help you with any bugs, questions, or contributions while you wait for a human maintainer. I'm an expert bot designed for just that!

To resolve the error message "[vertex_ai] Bad Request Error, 400 Please ensure that multiturn requests alternate between user and model" when using the parameter extractor in the parameter extraction node with the Claude3 series model or the Gemini series model, you need to ensure that the requests alternate correctly between the user and the model. This means that each request should follow the pattern of user input followed by model response.

Here are the steps to configure the parameter extractor correctly:

Select the Input Variable: Ensure that the input variable is correctly set up for parameter extraction.
Choose the Model: Select either the Claude3 series model or the Gemini series model.
Define the Parameters to Extract: Manually add or quickly import the parameters from existing tools.
Write Instructions: Provide clear instructions and examples to help the LLM improve the effectiveness and stability of extracting complex parameters.
Ensure Alternating Requests: Make sure that the requests alternate between user input and model response. This can be done by structuring the conversation flow properly.

Example Configuration:

{
  "input_variable": "user_query",
  "model": "Claude3",
  "parameters_to_extract": ["paper_id"],
  "instructions": "Extract the paper ID from the user's query.",
  "alternating_requests": true
}

Additionally, you can refer to the integration tests for the parameter extractor node to understand how to set up and test the configuration. For example, the test_chat_parameter_extractor function in the integration tests demonstrates how to configure and run the parameter extractor node with the Claude3 model:

def test_chat_parameter_extractor(setup_anthropic_mock):
    node = ParameterExtractorNode(
        tenant_id='1',
        app_id='1',
        workflow_id='1',
        user_id='1',
        invoke_from=InvokeFrom.WEB_APP,
        user_from=UserFrom.ACCOUNT,
        config={
            'id': 'llm',
            'data': {
                'title': '123',
                'type': 'parameter-extractor',
                'model': {
                    'provider': 'anthropic',
                    'name': 'claude-2',
                    'mode': 'chat',
                    'completion_params': {}
                },
                'query': ['sys', 'query'],
                'parameters': [{
                    'name': 'location',
                    'type': 'string',
                    'description': 'location',
                    'required': True
                }],
                'reasoning_mode': 'prompt',
                'instruction': '',
                'memory': None,
            }
        }
    )

    node._fetch_model_config = get_mocked_fetch_model_config(
        provider='anthropic', model='claude-2', mode='chat', credentials={
            'anthropic_api_key': os.environ.get('ANTHROPIC_API_KEY')
        }
    )
    db.session.close = MagicMock()

    # construct variable pool
    pool = VariablePool(system_variables={
        SystemVariable.QUERY: 'what\'s the weather in SF',
        SystemVariable.FILES: [],
        SystemVariable.CONVERSATION_ID: 'abababa',
        SystemVariable.USER_ID: 'aaa'
    }, user_inputs={})

    result = node.run(pool)

    assert result.status == WorkflowNodeExecutionStatus.SUCCEEDED
    assert result.outputs.get('location') == ''
    assert result.outputs.get('__reason') == 'Failed to extract result from function call or text response, using empty result.'
    prompts = result.process_data.get('prompts')

    for prompt in prompts:
        if prompt.get('role') == 'user':
            if '<structure>' in prompt.get('text'):
                assert '<structure>\n{"type": "object"' in prompt.get('text')

By following these steps and ensuring that the requests alternate between user and model, you should be able to resolve the error ^[1]^[2].

To continue talking to Dosu, mention @dosu.

langgenius / dify