[BUG] [SP-API] [Issue] AssumeRole request from Python returns SignatureDoesNotMatch

[tenbasetom]

I'm submitting a...

[ ] Regression (a behavior that used to work and stopped working in a new release)
[x ] Bug report  
[ ] Feature request
[ ] Documentation issue or request
[ ] Other... Please describe:

When making a request to /products/pricing/v0/competitivePrice we get a 403 error. This is strange because we are able to make requests to the /reports/2021-06-30/reports URI just fine.

Function to make the call:

[REDACTED]

This WORKS:

start = TIMESTAMP + timedelta(days=-30)
report = {
  'method': 'POST',
  'uri': '/reports/2021-06-30/reports',
  'params': {
    'marketplaceIds' : [MARKET_ID],
    'reportType' : 'GET_FLAT_FILE_ALL_ORDERS_DATA_BY_ORDER_DATE_GENERAL',
    'dataStartTime' : start.isoformat(),
    'dataEndTime' : TIMESTAMP.isoformat(),
  },
}
report_response = sp_api(report['method'], report['uri'], '', report['params'])
print(report_response.json())

This does NOT work (403 error):

ASIN = '...'
price = {
  'method': 'GET',
  'uri': '/products/pricing/v0/competitivePrice',
  'params': f'MarketplaceId={MARKET_ID}&ItemId={ASIN}&ItemType=Asin',
}
price_response = sp_api(price['method'], price['uri'], price['params'], '')
print(price_response.json())

It seems POST requests work, but GET requests do not work.

SP-API Name and Version or Report/Feed Type

Product Pricing v0 https://developer-docs.amazon.com/sp-api/docs/product-pricing-api-v0-use-case-guide

Expected Behavior

It should return the competitivePrice JSON as documented in the link above.

Current Behavior

It returns a 403 response.

Context/Screenshots

I'm trying to get the competitivePrice data for an ASIN.

Your Environment

Windows 10 Python 3.11.2 App has all roles checked in Seller Central

[rugved1991]

Hi @tenbasetom,

Thanks for reaching out. Can you please confirm if you have either "Product Listing" or "Pricing" role enabled on your application? The getCompetitivePricing operation is accessible if you have access to one of these two roles.
Once you have added these roles to your application and have reauthorized the app, you can test again and confirm if you are able to make a call. If the issue persists, please open a developer support case to investigate the issue further.

Best, Rugved Solutions Architect, SP API

[sungolivia]

I was gonna suggest same thing as @rugved1991 let me know when you check @tenbasetom 😄 also attaching roles blog that might help in case! https://developer-docs.amazon.com/sp-api-blog/docs/choosing-roles-for-developer-profile-and-sp-api-app

[tenbasetom]

@rugved1991 @sungolivia I have all roles checked for this app:

[tenbasetom]

UPDATE: Now both /products/pricing/v0/competitivePrice and /reports/2021-06-30/reports endpoints are returning 403 errors. What is going on? This is not a good developer experience.

UPDATE 2: I just ran /reports/2021-06-30/reports again and now I'm getting a 202 error.

[rugved1991]

Hi @tenbasetom,

Are you using a newly generated refresh token to get the access token? If the issue persists, please open a developer support case with complete request response and headers to investigate the issue further. Please provide case ID here so that we can check-in to see progress.

Best, Rugved Solutions Architect, SP API

[rugved1991]

Hi @tenbasetom,

Thanks for sharing. The grant_type mentioned here is "refresh_token". This means you are using the existing refresh token to get a new set of access token. When we need to reauthorize the application, you would need to follow the "authorization_code" grant flow once again to get a new set of refresh and access token. This would allow you to get authorization for the latest set of SP API roles associated with the application. If you are self-authorizing the application, please generate a new fresh refresh token from your seller central account and use that to get a new access token. I hope this information helps. If you are still seeing the same issue, please open a developer support case and we can investigate the issue further.

Best, Rugved Solutions Architect, SP API

[tenbasetom]

@rugved1991 thank you for replying, I will try as you instructed next. However, I'm curious about these things, so hope you have some insights:

1. How come I was able to call the /reports/2021-06-30/reports endpoint before using my existing app (dramazon3)? This tells me I'm already authorized on SP API.
2. Why did the /reports/2021-06-30/reports endpoint suddenly stop working yesterday?
3. Can you explain what might cause the /reports/2021-06-30/reports endpoint to return 403, then one minute ater return 202? Is Amazon currently in the middle of migrating something that might suddenly cause things to stop working like this?

The only thing I can think of is that AWS has a limit of FOUR roles, so I must have accidentally deleted a role that was required when I added a 5th role two days ago. Now I don't know how to get back the role that was deleted by AWS. I wish AWS would give us a warning before it does something destructive like deleting a role.

[tenbasetom]

UPDATE: The /reports/2021-06-30/reports endpoint works now! Do you know what's going on? Why does the API seem so unstable? 403 -> 202 > 200

UPDATE 2: The /products/pricing/v0/competitivePrice endpoint still doesn't work. I'm trying to get more detailed info about the cause but it seems the 403 response is not in JSON format.

# response = <Response [403]>
response_json = response.json()
print(response_json)

Traceback (most recent call last):
  File "path\to\Python\Python311\Lib\site-packages\requests\models.py", line 971, in json
    return complexjson.loads(self.text, **kwargs)    
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^    
  File "path\to\Python\Python311\Lib\json\__init__.py", line 346, in loads  
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "path\to\Python\Python311\Lib\json\decoder.py", line 337, in decode  
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "path\to\Python\Python311\Lib\json\decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
               ^^^^^^^^^^^^^^^^^^^^^^
json.decoder.JSONDecodeError: Invalid control character at: line 4 column 197 (char 218)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "path\to\test.py", line 117, in get_pricing
    response_json = response.json()    
                           ^^^^^^^^^^^^^^^^^^^^^^    
  File "path\to\Python\Python311\Lib\site-packages\requests\models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Invalid control 
character at: line 4 column 197 (char 218)

[tenbasetom]

Btw, here's the case #: 12409012651

However, Amazon dev support has been non-responsive so far :(. Once again, our app already has the Pricing role (as you can see in the screenshot above), so in theory it should be able to use the /products/pricing/v0/competitivePrice endpoint, right? I don't know what else we can do.

https://developer-docs.amazon.com/sp-api/docs/pricing-role

UPDATE 3: After more research, it seems that there are TWO places where we need to have the proper roles: app roles (which all is checked), and developer profile roles (which we don't know where to check since I don't remember what was checked when the profiles was first created).

Questions:

1. How do I know what role my developer profile has?
2. How do I add new roles to my developer profile?

[rugved1991]

Hi @tenbasetom ,

Thanks for creating a developer support case. Please add complete request and response examples showing the error along with headers. This would allow the developer support team to investigate the issue further. To answer your question about roles:

1. You can view developer profile here
2. You can select or deselect any role you wish on the developer profile. If you select any restricted role, you will have to provide additional information about how you would handle the Amazon customer data.

I hope this information helps. Please follow up on the support case for further investigation.

Best, Rugved Solutions Architect, SP API

[tenbasetom]

@rugved1991 thanks, I just checked developer profile and here's what I see:

Okay, I will go update the case now. I just tried another API endpoint (/listings/2021-08-01/items/{seller_id}/{sku}) and it's also returning 403 :(

[rugved1991]

Hi @tenbasetom,

Can you please confirm if you are able to test the API calls using Postman or curl? This seems like a code issue rather than an API issue. Lets confirm if you are able to call the API successfully through other tools. The above response snippet doesnt help to investigate further as there is no requestID or timestamp to troubleshoot.

Best, Rugved Solutions Architect, SP API

[tenbasetom]

@rugved1991 Postman also fails.

"This seems like a code issue rather than an API issue." -- If it's a code issue, then can you please explain why /reports/2021-06-30/reports endpoint works? It's a POST with a body that is a more complex than the GET /products/pricing/v0/competitivePrice endpoint.

If it makes things easier for you to troubleshoot, I can provide you the Python script and you can see for yourself the issues I'm having. We have been using this script for a long time to generate reports without issues. It's only when I tried to make a request to /products/pricing/v0/competitivePrice that I started experiencing issues.

[rugved1991]

Hi @tenbasetom,

Please add complete request and response details from your Postman test on the support case. The Console log in Postman will have all the details required to investigate the issue further. We wont be able to debug your python code to investigate the issue. We need the request ID and timestamp from the response headers to perform a log dive on our end. I was able to successfully get a response on Postman with the following request:

https://sellingpartnerapi-na.amazon.com/products/pricing/v0/competitivePrice?MarketplaceId=ATVPDKIKX0DER&ItemType=Asin&Asins=B009F3PX2S

I hope this helps.

Best, Rugved Solutions Architect, SP API

[tenbasetom]

@rugved1991 We finally got Postman to work, and now I'm trying to replicate the calls I have in Postman to Python, but when I call the AssumeRole action I keep getting the "SignatureDoesNotMatch" error. I've tried all these combinations to create the signature so far, but nothing works...

  method = 'GET'
  uri = '/'
  query_string = f'Action=AssumeRole&RoleArn={urllib.parse.quote(AWS_ROLE_ARN)}&RoleSessionName=PythonYoyo&Version={STS_VERSION}'
  payload = ''

  # Try 1
  canonical_headers = f'host:{STS_ENDPOINT}\nx-amz-date:{date_time}'
  signed_headers = 'host;x-amz-date'

  # Try 2
  canonical_headers = f'host:{STS_ENDPOINT}\nx-amz-access-token:{access_token}\nx-amz-date:{date_time}'
  signed_headers = 'host;x-amz-access-token;x-amz-date'

  # Try 3
  canonical_headers = f'content-type:{URLENCODED_CONTENT_TYPE}\nhost:{STS_ENDPOINT}\nx-amz-access-token:{access_token}\nx-amz-date:{date_time}'
  signed_headers = 'content-type;host;x-amz-access-token;x-amz-date'

Results for try amzn/selling-partner-api-models#672...

Request URL:

GET https://sts.amazonaws.com/?Action=AssumeRole&RoleArn=arn%3Aaws%3Aiam%3A%3A665728163257%3Arole/DrAmazonRoleName&RoleSessionName=PythonYoyo&Version=2011-06-15

Request headers:

{'content-type': 
'application/x-www-form-urlencoded; charset=utf-8', 'x-amz-access-token': 'Atza|IwEBILRXh4faEzAtyfskofLGdJaI4gOKLEuLa4Ux0Bb59b5CszIpC3xVoyAZ0Gp_cIQvip8qQgriFsj-EbECRufIMD-nrNRS4LuPdyCo6vDkV7Ir-75HDeOe_OLyBeqmnioE8fbc2-awhWqEgDiVvv8jWz3X3j84NQFf2kyfuN_5XUo9pDhMoiURr6WLnxqgQLOOtXRAbQ9OmuUUh2vrTnvzkIQMM6dztEvZ8f_F0N7uOkwhTUC5U-DOK7GVPbzhrhYnMdb68yBMC3Wa-VZVSazjyuG-FaVgrdQUMj-NmdWqKAX_dYIR2FJ-ZHl8ntP5gZM9t67cITDdkgrE7LBAqOgTSbbq', 'x-amz-date': '20230407T053700Z', 'authorization': 'AWS4-HMAC-SHA256 Credential=AKIAZWAD5TW4V2GOXW4Q/20230407/us-east-1/sts/aws4_request, SignedHeaders=content-type;host;x-amz-access-token;x-amz-date, Signature=60fb3e7820971a81a766ba5da8f678009e025b3bbecf0536cf625da64cc2458f'}

Response:

{'_content': b'<ErrorResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">\n  <Error>\n    <Type>Sender</Type>\n    <Code>SignatureDoesNotMatch</Code>\n    <Message>The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.</Message>\n  </Error>\n  <RequestId>b1189afb-ba5f-4dee-9ec1-24688a45015f</RequestId>\n</ErrorResponse>\n', '_content_consumed': True, '_next': None, 'status_code': 403, 'headers': {'x-amzn-RequestId': 'b1189afb-ba5f-4dee-9ec1-24688a45015f', 'Content-Type': 'text/xml', 'Content-Length': '431', 'Date': 'Fri, 07 Apr 2023 05:36:32 GMT'}, 'raw': <urllib3.response.HTTPResponse object at 0x000002BB66C61D50>, 'url': 'https://sts.amazonaws.com/?Action=AssumeRole&RoleArn=arn%3Aaws%3Aiam%3A%3A665728163257%3Arole/DrAmazonRoleName&RoleSessionName=PythonYoyo&Version=2011-06-15', 'encoding': 'ISO-8859-1', 'history': [], 'reason': 'Forbidden', 
'cookies': <RequestsCookieJar[]>, 'elapsed': datetime.timedelta(microseconds=464117), 'request': <PreparedRequest [GET]>, 'connection': <requests.adapters.HTTPAdapter object 
at 0x000002BB66C5FB90>}

[tenbasetom]

UPDATE: I found out the root cause: urllib.parse.quote() was not correctly URL-encoding the ARN:

RoleArn=arn%3Aaws%3Aiam%3A%3A665728163257%3Arole/DrAmazonRoleName

Since the "/" was not encoded, it was resulting in a signature mismatch, hence the error.