InvalidRequestException while using Comprehend DetectToxicContentCommand

raphaelbharel commented 3 months ago

Checkboxes for prior research

[X] I've gone through Developer Guide and API reference
[X] I've checked AWS Forums and StackOverflow.
[X] I've searched for previous similar issues and didn't find any solution.

Describe the bug

error while using the Comprehend DetectToxicContentCommand endpoint via the ComprehendClient.

Following error is observed:

2024-04-07T16:04:41.079000+00:00 2024/04/07/[$LATEST]17cf4ac292f6499b8c701637d0c0de0e {"level":"ERROR","message":"Error detecting toxic content: {\"name\":\"InvalidRequestException\",\"$fault\":\"client\",\"$metadata\":{\"httpStatusCode\":400,\"requestId\":\"7aac7ab8-87c5-4906-ac72-ad6e9dda91a6\",\"attempts\":1,\"totalRetryDelay\":0},\"__type\":\"InvalidRequestException\"}","service":"service_undefined","timestamp":"2024-04-07T16:04:41.078Z","xray_trace_id":"1-6612c418-d35f03d99496d1343894aaac"}

SERVERLESS IAM permission:

Effect: 'Allow' Action:
comprehend: Resource:

SDK version number

@aws-sdk/client-comprehend: 3.549.0

Which JavaScript Runtime is this issue in?

Node.js

Details of the browser/Node.js/ReactNative version

Node.js v18.17.0

Reproduction Steps

Create an AWS Comprehend lambda endpoint with the following configuration:

const messageString = 'Hello world';
  const textSegments = [{ Text: messageString }];

  const detectToxicContentParams = {
    LanguageCode: LanguageCode.EN,
    TextSegments: textSegments,
  };

  try {
    logger.info(`Input params: ${JSON.stringify(detectToxicContentParams)}`);

    const detectToxicContentResult = await comprehendClient.send(
      new DetectToxicContentCommand(detectToxicContentParams)
    );

    if (!detectToxicContentResult.ResultList || detectToxicContentResult.ResultList.length === 0) {
      throw new Error('No result returned from Comprehend');
    }

    const resultList = detectToxicContentResult.ResultList[0] as ToxicLabels;
    if (resultList.Toxicity === undefined) {
      throw new Error('No toxicity score returned from Comprehend');
    }

    if (resultList.Toxicity > 0.5) {
      return {
        statusCode: 400,
        body: JSON.stringify({ error: 'Message contains toxic content' }),
      };
    }
  } catch (error) {
    logger.error(`Error detecting toxic content: ${JSON.stringify(error)}`);
    return {
      statusCode: 500,
      body: JSON.stringify({ error: 'Internal Server Error' }),
    };
  }

Observed Behavior

2024-04-07T16:04:41.079000+00:00 2024/04/07/[$LATEST]17cf4ac292f6499b8c701637d0c0de0e {"level":"ERROR","message":"Error detecting toxic content: {\"name\":\"InvalidRequestException\",\"$fault\":\"client\",\"$metadata\":{\"httpStatusCode\":400,\"requestId\":\"7aac7ab8-87c5-4906-ac72-ad6e9dda91a6\",\"attempts\":1,\"totalRetryDelay\":0},\"__type\":\"InvalidRequestException\"}","service":"service_undefined","timestamp":"2024-04-07T16:04:41.078Z","xray_trace_id":"1-6612c418-d35f03d99496d1343894aaac"}

Expected Behavior

no thrown error, as was observed when using the DetectSentimentCommand

Possible Solution

Maybe an explicit IAM permission for the DetectToxicityCommand ?

Additional Information/Context

No response

RanVaknin commented 3 months ago

Hi @raphaelbharel ,

I can run this command from my local node.js environment successfully:

import { ComprehendClient, DetectToxicContentCommand, LanguageCode } from "@aws-sdk/client-comprehend";

const client = new ComprehendClient({
  region: "us-east-1"
})
const messageString = 'Hello world';
  const textSegments = [{ Text: messageString }];

try {
  const response = await client.send(new DetectToxicContentCommand({
    LanguageCode: LanguageCode.EN,
    TextSegments: textSegments,
  }))
  console.log(response)
} catch (error) {
  console.log(error)

}

Response:

{
  '$metadata': {
    httpStatusCode: 200,
    requestId: '0a23589a-REDACTED',
    extendedRequestId: undefined,
    cfId: undefined,
    attempts: 1,
    totalRetryDelay: 0
  },
  ResultList: [ { Labels: [Array], Toxicity: 0.08969999849796295 } ]
}

You stated you are using @aws-sdk/client-comprehend: 3.549.0, and also that you are using Lambda. Are you providing your own SDK as a lambda layer of bundled your application with this version and uploaded it to lambda? I'm asking because the Lambda provided SDK is not pinned to aws-sdk/client-comprehend@latest. The Lambda service team only updates their provided SDK versions a few times a year.

Looking at DetectToxicContentCommand this was introduced by the Comprehend service team 5 months ago, which leads me to believe that the Lambda provided SDK version is older than when this command was released.

̶U̶n̶f̶o̶r̶t̶u̶n̶a̶t̶e̶l̶y̶ ̶L̶a̶m̶b̶d̶a̶ ̶h̶a̶s̶ ̶m̶a̶d̶e̶ ̶i̶t̶ ̶h̶a̶r̶d̶e̶r̶ ̶t̶o̶ ̶k̶n̶o̶w̶ ̶w̶h̶i̶c̶h̶ ̶e̶x̶a̶c̶t̶ ̶S̶D̶K̶ ̶v̶e̶r̶s̶i̶o̶n̶ ̶i̶s̶ ̶b̶e̶i̶n̶g̶ ̶p̶r̶o̶v̶i̶d̶e̶d̶, but my suggestion to you is to provide your own SDK to lambda.

**Edit: The Lambda team has now introduced a way to see which SDK version is being provided: https://docs.aws.amazon.com/lambda/latest/dg/lambda-nodejs.html#nodejs-sdk-included

Let me know what you find. Thanks, Ran~

github-actions[bot] commented 2 months ago

This issue has not received a response in 1 week. If you still think there is a problem, please leave a comment to avoid the issue from automatically closing.

github-actions[bot] commented 2 months ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.

aws / aws-sdk-js-v3