AWS Bedrock errors with follow up conversation messages when using fromDecision

[airtonix]

Playing around with this as a gitsubmodule in my private repo and after finally getting vercels ai sdk to connect to bedrock, I think I've stumbled into an issue around how your agent assumes it can provide message memory when under fromDecision execution mode.

A conversation must start with a user message. Try again with a conversation that starts with a user message.

[!NOTE] @xstate/agent here is whatever your mainline branch is now:

import { z } from 'zod';
import { createAgent, fromDecision } from '@xstate/agent';
import { fromPromise, assign, createActor, setup } from 'xstate';
import { loadSetting } from '@ai-sdk/provider-utils';
import { createAmazonBedrock } from '@ai-sdk/amazon-bedrock';

const getFromTerminal = fromPromise<string, string>(async ({ input }) => {
  const topic = await new Promise<string>((res) => {
    console.log(input + '\n');
    const listener = (data: Buffer) => {
      const result = data.toString().trim();
      process.stdin.off('data', listener);
      res(result);
    };
    process.stdin.on('data', listener);
  });

  return topic;
});

const createBedrockModel = createAmazonBedrock({
  bedrockOptions: {
    region: loadSetting({
      settingValue: undefined,
      settingName: 'region',
      environmentVariableName: 'AWS_REGION',
      description: 'AWS region',
    }),
    credentials: {
      accessKeyId: loadSetting({
        settingValue: undefined,
        settingName: 'accessKeyId',
        environmentVariableName: 'AWS_ACCESS_KEY_ID',
        description: 'AWS access key ID',
      }),
      secretAccessKey: loadSetting({
        settingValue: undefined,
        settingName: 'secretAccessKey',
        environmentVariableName: 'AWS_SECRET_ACCESS_KEY',
        description: 'AWS secret access key',
      }),
      sessionToken: loadSetting({
        settingValue: undefined,
        settingName: 'sessionToken',
        environmentVariableName: 'AWS_SESSION_TOKEN',
        description: 'AWS secret access key',
      }),
    },
  },
});

const agent = createAgent({
  name: 'email',
  model: createBedrockModel('anthropic.claude-3-haiku-20240307-v1:0'),
  events: {
    askForClarification: z.object({
      questions: z.array(z.string()).describe('The questions to ask the agent'),
    }),
    submitEmail: z.object({
      email: z.string().describe('The email to submit'),
    }),
  },
});

const machine = setup({
  types: {
    events: agent.eventTypes,
    input: {} as {
      email: string;
      instructions: string;
    },
    context: {} as {
      email: string;
      instructions: string;
      clarifications: string[];
      replyEmail: string | null;
    },
  },
  actors: { agent: fromDecision(agent), getFromTerminal },
}).createMachine({
  initial: 'checking',
  context: (x) => ({
    email: x.input.email,
    instructions: x.input.instructions,
    clarifications: [],
    replyEmail: null,
  }),
  states: {
    checking: {
      invoke: {
        src: 'agent',
        input: (x) => ({
          context: {
            email: x.context.email,
            instructions: x.context.instructions,
            clarifications: x.context.clarifications,
          },
          messages: agent.select((ctx) => {
            console.log(JSON.stringify(ctx.messages, null, 2));
            return ctx.messages.filter((m) => !m.content);
          }),
          goal: 'Respond to the email given the instructions and the provided clarifications. If not enough information is provided, ask for clarification. Otherwise, if you are absolutely sure that there is no ambiguous or missing information, create and submit a response email.',
        }),
      },
      on: {
        askForClarification: {
          actions: (x) => console.log(x.event.questions.join('\n')),
          target: 'clarifying',
        },
        submitEmail: {
          target: 'submitting',
        },
      },
    },
    clarifying: {
      invoke: {
        src: 'getFromTerminal',
        input: `Please provide answers to the questions above`,
        onDone: {
          actions: assign({
            clarifications: (x) =>
              x.context.clarifications.concat(x.event.output),
          }),
          target: 'checking',
        },
      },
    },
    submitting: {
      invoke: {
        src: 'agent',
        input: ({ context }) => ({
          context: {
            email: context.email,
            instructions: context.instructions,
            clarifications: context.clarifications,
          },
          goal: `Create and submit an email based on the instructions.`,
        }),
      },
      on: {
        submitEmail: {
          actions: assign({
            replyEmail: ({ event }) => event.email,
          }),
          target: 'done',
        },
      },
    },
    done: {
      type: 'final',
      entry: (x) => console.log(x.context.replyEmail),
    },
  },
  exit: () => {
    console.log('End of conversation.');
    process.exit();
  },
});

createActor(machine, {
  input: {
    email: 'That sounds great! When are you available?',
    instructions:
      'Tell them exactly when I am available. Address them by his full (first and last) name.',
  },
}).start();

1. I get first prompt: "what is the persons name"
2. I type a name in
3. it throws an error

❯ yarn tsx ./src/stately.ts
[]
What is the person's full name?
Please provide answers to the questions above

Joe Bloggs
[
  {
    "id": "REDACTED",
    "role": "user",
    "content": "<context>{\"email\":\"That sounds great! When are you available?\",\"instructions\":\"Tell them exactly when I am available. Address them by his full (first and last) name.\",\"clarifications\":[]}</context>\n\n<context>{\"email\":\"That sounds great! When are you available?\",\"instructions\":\"Tell them exactly when I am available. Address them by his full (first and last) name.\",\"clarifications\":[]}</context>\n\nRespond to the email given the instructions and the provided clarifications. If not enough information is provided, ask for clarification. Otherwise, if you are absolutely sure that there is no ambiguous or missing information, create and submit a response email.\n\nOnly make a single tool call to achieve the above goal.",
    "timestamp": "REDACTED_NUMBER",
    "sessionId": "x:0"
  },
  {
    "content": "",
    "id": "REDACTED",
    "role": "assistant",
    "timestamp": "REDACTED_NUMBER",
    "responseId": "REDACTED",
    "result": {
      "text": "",
      "toolCalls": [
        {
          "type": "tool-call",
          "toolCallId": "tooluse_REDACTED_ID_2",
          "toolName": "askForClarification",
          "args": {
            "questions": [
              "What is the person's full name?"
            ]
          }
        }
      ],
      "toolResults": [
        {
          "toolCallId": "tooluse_REDACTED_ID_2",
          "toolName": "askForClarification",
          "args": {
            "questions": [
              "What is the person's full name?"
            ]
          },
          "result": {
            "type": "askForClarification",
            "questions": [
              "What is the person's full name?"
            ]
          }
        }
      ],
      "finishReason": "tool-calls",
      "usage": {
        "promptTokens": 690,
        "completionTokens": 51,
        "totalTokens": 741
      },
      "warnings": [],
      "responseMessages": [
        {
          "role": "assistant",
          "content": [
            {
              "type": "text",
              "text": ""
            },
            {
              "type": "tool-call",
              "toolCallId": "tooluse_REDACTED_ID_2",
              "toolName": "askForClarification",
              "args": {
                "questions": [
                  "What is the person's full name?"
                ]
              }
            }
          ]
        },
        {
          "role": "tool",
          "content": [
            {
              "type": "tool-result",
              "toolCallId": "tooluse_REDACTED_ID_2",
              "toolName": "askForClarification",
              "result": {
                "type": "askForClarification",
                "questions": [
                  "What is the person's full name?"
                ]
              }
            }
          ]
        }
      ]
    },
    "sessionId": "x:0"
  }
]
/REDACTED_FILEPATH/node_modules/xstate/dist/raise-ff8990f7.cjs.js:125
    throw err;
    ^

ValidationException: A conversation must start with a user message. Try again with a conversation that starts with a user message.
    at de_ValidationExceptionRes (/REDACTED_FILEPATH/node_modules/@aws-sdk/client-bedrock-runtime/dist-cjs/index.js:1082:21)
    at de_CommandError (/REDACTED_FILEPATH/node_modules/@aws-sdk/client-bedrock-runtime/dist-cjs/index.js:937:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /REDACTED_FILEPATH/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20
    at async /REDACTED_FILEPATH/node_modules/@smithy/core/dist-cjs/index.js:165:18
    at async /REDACTED_FILEPATH/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38
    at async /REDACTED_FILEPATH/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:34:22
    at BedrockChatLanguageModel.doGenerate (/REDACTED_FILEPATH/node_modules/@ai-sdk/amazon-bedrock/src/bedrock-chat-language-model.ts:159:22)
    at _retryWithExponentialBackoff (/REDACTED_FILEPATH/node_modules/ai/core/util/retry-with-exponential-backoff.ts:36:12)
    at Object.generateText (/REDACTED_FILEPATH/node_modules/ai/core/generate-text/generate-text.ts:129:28) {
  '$fault': 'client',
  '$metadata': {
    httpStatusCode: 400,
    requestId: 'REDACTED_AWS_REQUEST_ID',
    extendedRequestId: undefined,
    cfId: undefined,
    attempts: 1,
    totalRetryDelay: 0
  }
}

Node.js v22.2.0

[davidkpiano]

Can you test this branch to see if it resolves the problem? https://github.com/statelyai/agent/pull/37

[airtonix]

Can you test this branch to see if it resolves the problem? #37

Thank you, I'll check it out tonight. 👍🏻