Aws::Pinpoint::Client create_import_job fails

[yeldah3]

Describe the bug

The create_import_job method on the Aws::Pinpoint::Client fails with the Aws::Pinpoint::Errors::BadRequestException

Unexpected character ('I' (code 73)): was expecting double-quote to start field name
 at [Source: (String)"{ Id: "9GSh1dnhtfjUTh8J9Yj9N33LBe4f9gn2x1zks" }"; line: 1, column: 4].

The stack trace points to lib/seahorse/client/plugins/raise_response_errors.rb:17

As far as I'm aware, this Id is part of the response payload from this request (looking at the docs here), and it's causing the sdk to fail in sending the response back to us.

Expected Behavior

A successful response

Current Behavior

The Aws::Pinpoint::Errors::BadRequestException is raised

Reproduction Steps

credentials = Aws::Credentials.new(<access_key_id>, <secret_key>)
client = Aws::Pinpoint::Client.new(region: <region>, credentials: credentials)
import_job_request = {
  define_segment: true,
  format: "JSON",
  register_endpoints: true,
  role_arn: <role_arn>,
  s3_url: <s3_url>,
  segment_name: <segment_name>
}

client.create_import_job(
  {
    application_id: <application_id>,
    import_job_request: import_job_request
  }
)

Possible Solution

No response

Additional Information/Context

No response

Gem name ('aws-sdk', 'aws-sdk-resources' or service gems like 'aws-sdk-s3') and its version

aws-sdk-pinpoint 1.68.0, with aws-sdk-core 3.127.0 (although I tried upgrading to aws-sdk-pinpoint 1.84.0/aws-sdk-core 3.188.0 and saw the same error).

Environment details (Version of Ruby, OS environment)

ruby 3.2.2 (2023-03-30 revision e51014f9c0) +YJIT [arm64-darwin22]

[mullermp]

Thanks for opening up an issue. Could you please provide a full stack trace and the wire response from the service? Set the Client option http_wire_trace: true (please remove your sensitive credentials from the response) to share the headers and share the response body by adding this plugin:

class PrintHttpBody < Seahorse::Client::Plugin
  class Handler < Seahorse::Client::Handler
    def call(context)
      puts "HTTP REQUEST BODY:"
      puts context.http_request.body.read
      context.http_request.body.rewind
      @handler.call(context)
    end
  end
  def add_handlers(handlers, cfg)
    handlers.add(Handler, step: :sign, priority: 99)
  end
end  
Aws::Pinpoint::Client.add_plugin(PrintHttpBody)

[yeldah3]

Thanks for opening up an issue. Could you please provide a full stack trace and the wire response from the service? Set the Client option http_wire_trace: true (please remove your sensitive credentials from the response) to share the headers and share the response body by adding this plugin:

class PrintHttpBody < Seahorse::Client::Plugin
  class Handler < Seahorse::Client::Handler
    def call(context)
      puts "HTTP REQUEST BODY:"
      puts context.http_request.body.read
      context.http_request.body.rewind
      @handler.call(context)
    end
  end
  def add_handlers(handlers, cfg)
    handlers.add(Handler, step: :sign, priority: 99)
  end
end  
Aws::Pinpoint::Client.add_plugin(PrintHttpBody)

Thanks Matt! Here's the output -

HTTP REQUEST BODY:
{"DefineSegment":true,"Format":"JSON","RegisterEndpoints":true,"RoleArn":"<roleArn>","S3Url":"s3://<bucket>/dummy","SegmentName":"boop"}
opening connection to pinpoint.us-east-2.amazonaws.com:443...
opened
starting SSL for pinpoint.us-east-2.amazonaws.com:443...
SSL established, protocol: TLSv1.3, cipher: TLS_AES_128_GCM_SHA256
<- "POST /v1/apps/<application_id>/jobs/import HTTP/1.1\r\nAccept-Encoding: \r\nUser-Agent: aws-sdk-ruby3/3.170.0 ruby/3.2.2 arm64-darwin22 aws-sdk-pinpoint/1.68.0\r\nContent-Type: application/json\r\nHost: pinpoint.us-east-2.amazonaws.com\r\nX-Amz-Date: 20231202T183353Z\r\nX-Amz-Content-Sha256: 0c8b42ab948f60d4f1b531ca918ec00d707c601c993bca12782fa247ebdca889\r\nAuthorization: AWS4-HMAC-SHA256 Credential=<access_key_id>/20231202/us-east-2/mobiletargeting/aws4_request, SignedHeaders=content-type;host;x-amz-content-sha256;x-amz-date, Signature=<signature>\r\nContent-Length: 213\r\nAccept: */*\r\nSentry-Trace: 0414711fe50f4cc093e9a149f7d50ba1-456648d14b5f4d61\r\nBaggage: sentry-trace_id=0414711fe50f4cc093e9a149f7d50ba1,sentry-environment=development,sentry-public_key=<public_key>\r\n\r\n"
<- "{\"DefineSegment\":true,\"Format\":\"JSON\",\"RegisterEndpoints\":true,\"RoleArn\":\"<roleArn>\",\"S3Url\":\"s3://<bucket>/dummy\",\"SegmentName\":\"boop\"}"
-> "HTTP/1.1 400 Bad Request\r\n"
-> "Content-Type: application/json\r\n"
-> "Content-Length: 227\r\n"
-> "Connection: keep-alive\r\n"
-> "Date: Sat, 02 Dec 2023 18:33:54 GMT\r\n"
-> "x-amzn-RequestId: 61ba6023-1556-4324-abf4-d13eb23ede9a\r\n"
-> "Access-Control-Allow-Origin: *\r\n"
-> "x-amzn-ErrorType: BadRequestException\r\n"
-> "x-amz-apigw-id: PU_G2GXIiYcEpuQ=\r\n"
-> "Cache-Control: no-store\r\n"
-> "X-Amzn-Trace-Id: Root=1-656b7891-33c6d52368f04e962f4936fc;Sampled=0;lineage=95aed0fd:0\r\n"
-> "X-Cache: Error from cloudfront\r\n"
-> "Via: 1.1 eb24f3949a23944cb67ad5e7353bdfb2.cloudfront.net (CloudFront)\r\n"
-> "X-Amz-Cf-Pop: SFO53-P4\r\n"
-> "X-Amz-Cf-Id: HmrorZ8IfoSRI_E48Qs-ynS_zMdlLsMfBa7SQeEWuuJdqOGJJbh8cA==\r\n"
-> "\r\n"
reading 227 bytes...
-> "{\"RequestID\":\"PU_G2GXIiYcEpuQ=\",\"Message\":\"Unexpected character ('I' (code 73)): was expecting double-quote to start field name\\n at [Source: (String)\\\"{ Id: \\\"9GSh1dnhtfjUTh8J9Yj9N33LBe4f9gn2x1zks\\\" }\\\"; line: 1, column: 4]\"}\n"
read 227 bytes
Conn keep-alive
/nix/store/li8nghi7sgqzwwz8hwrp3s06qlcw98m7-ruby3.2.2-aws-sdk-core-3.170.0/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.170.0/lib/seahorse/client/plugins/raise_response_errors.rb:17:in `call': Unexpected character ('I' (code 73)): was expecting double-quote to start field name (Aws::Pinpoint::Errors::BadRequestException)
 at [Source: (String)"{ Id: "9GSh1dnhtfjUTh8J9Yj9N33LBe4f9gn2x1zks" }"; line: 1, column: 4]

[mullermp]

Thanks for providing that. Based on the wire log, it looks like the error message is definitely from the service, and not caused from SDK parsing. The SDK is parsing that message correctly into BadRequestException. I am going to forward this to our support team and open a ticket with the service team.

[yeldah3]

Awesome, thank you so much!

[RanVaknin]

Hi @yeldah3 ,

I have tried to reproduce this using the Go SDK v2 and got a similar result. I have created an internal ticket with the Pinpoint team for further investigation.

Will keep you updated.

Thanks, Ran~

P107759825

[RanVaknin]

Hi @yeldah3,

I heard back from the Pinpoint service team. My problem was that the input I provided was a json object that was new line formatted. When I change the json file to a single line input:

{"Address":"example@email.com","ChannelType":"EMAIL","Attributes":{"CustomAttribute1":["value1","value2"],"CustomAttribute2":["value3","value4"]},"Demographic":{"AppVersion":"1.0","Locale":"en_US","Make":"Apple","Model":"iPhone","ModelVersion":"X","Platform":"ios"},"Location":{"Country":"US","Latitude":47.615556,"Longitude":-122.334444,"PostalCode":"98101","City":"Seattle"},"Metrics":{"CustomMetric1":1.0,"CustomMetric2":2.0},"User":{"UserId":"user123","UserAttributes":{"Interests":["sports","music"],"Age":["30"]}}}

This fixed the issue for me.

Can you check the JSON file you uploaded to S3 to make sure it conforms to format highlighted above?

Additionally, please check this documentation https://docs.aws.amazon.com/pinpoint/latest/userguide/segments-importing.html

Thanks, Ran~

[yeldah3]

Hey Ran! Good catch, the issue was in fact that our json object was not properly formatted. We've fixed it on our end and not get 200s back from the API. Thanks for your help!

[RanVaknin]

anytime :)

[github-actions[bot]]

This issue is now closed.

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.