Feeds API, feed processing report shows: "your uploaded document has an illegal block size"

[fmeccanici]

Dear all,

I am working on updating order shipping information using the Feeds API, which I am coding in PHP using https://github.com/clousale/amazon-sp-api-php. The problem is after I upload the document (response shows status code 200 OK) and creating a feed, the feed processing status becomes 'FATAL' instead of 'DONE'. Decrypting the processing report shows the following error: "The uploaded document has an illegal block size".

To achieve encryption and upload the encrypted XML I use the following PHP code:

$feedDocument = $feedsApi->createFeedDocument(new \ClouSale\AmazonSellingPartnerAPI\Models\Feeds\CreateFeedDocumentSpecification([
    'content_type' => 'application/xml; charset=UTF-8'
]));

$documentId = $feedDocument->getPayload()->getFeedDocumentId();
$url = $feedDocument->getPayload()->getUrl();
$key = $feedDocument->getPayload()->getEncryptionDetails()->getKey();
$key = base64_decode($key, true);
$initializationVector = $feedDocument->getPayload()->getEncryptionDetails()->getInitializationVector();
$initializationVector = base64_decode($initializationVector, true);

$file = file_get_contents('./files/order_feed.xml');
$feedData = utf8_encode($file);

$feedDataEncrypted = AESCryptoStreamFactory::encrypt($feedData, $key, $initializationVector);

$api = new RestClient();
$response = $api->put($url, [$feedDataEncrypted], ['Content-Type' => 'application/xml; charset=UTF-8']);

The response gives me an HTTP response code of 200 OK, which suggests that everything is fine. My XML looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<AmazonEnvelope xsi:noNamespaceSchemaLocation="amzn-envelope.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <Header>
        <DocumentVersion>1.01</DocumentVersion>
        <MerchantIdentifier>A2L0**********</MerchantIdentifier>
    </Header>
    <MessageType>OrderFulfillment</MessageType>
    <Message>
        <MessageID>1</MessageID>
        <OperationType>Update</OperationType>
        <OrderFulfillment>
            <AmazonOrderId>405-*******-*******</AmazonOrderId>
            <FulfillmentDate>2021-02-02T10:40:00</FulfillmentDate>
            <FulfillmentData>
                <CarrierName>Post NL</CarrierName>
                <ShippingMethod>Standard</ShippingMethod>
                <ShipperTrackingNumber>3SYUDM********</ShipperTrackingNumber>
            </FulfillmentData>
            <Item>
                <AmazonOrderItemCode>0232392*******</AmazonOrderItemCode>
                <Quantity>1</Quantity>
            </Item>
        </OrderFulfillment>
    </Message>
</AmazonEnvelope>

And the encrypt function looks like this:

public const BLOCK_SIZE = 8;
public const IV_LENGTH = 16;
public const CIPHER = 'AES256';

public static function encrypt(string $plainText, string $key, string $iv): string
{
    $plainText = static::getPaddedText($plainText);
    return openssl_encrypt($plainText, static::CIPHER, $key, OPENSSL_RAW_DATA, $iv);
}

protected static function getPaddedText(string $plainText): string
{
    $stringLength = strlen($plainText);
    if ($stringLength % static::BLOCK_SIZE) {
        $plainText = str_pad($plainText, $stringLength + static::BLOCK_SIZE - $stringLength % static::BLOCK_SIZE, "\0");
    }
    return $plainText;
}

Does someone know what's going wrong? I have been stuck on this for quite a while. Thanks a lot for your help!

[iosdevel]

Any luck figuring this out?

[loct991]

Same problem here. Did anyone make this work? Any help is greatly appreciated

[stefnats]

Is the encrypt function well tested? What is the source of the encrypt code (which repo)?

[iosdevel]

As a note, if using the Clousale repo, this will not work. I tried it with the double-break repo and it all works well.

[HK-khandh]

fmeccanici@ were you able to resolve this? or get any guidance here?

[cristoper]

public const BLOCK_SIZE = 8;

Doesn't AES always use 16-byte blocks?

[nightcoding2021]

Did anyone on this thread figure out this problem? I keep getting "The uploaded document has an illegal block size."

[cristoper]

@nightcoding2021 I think that error usually indicates the encrypted document is being padded incorrectly. If you share your code someone might be able to spot the problem.

[nightcoding2021]

Here is the code. strData is the file contents being uploaded. strKey and strIV are the unaltered base64 strings as returned by the call to createFeedDocument...

Public Function AES_Encrypt(ByVal strData As String, ByVal strKey As String, ByVal strIV As String) As Byte()
    Try
        Dim aes As New RijndaelManaged
        aes.KeySize = 256

        Dim key() As Byte = Convert.FromBase64String(strKey)
        Dim IV() As Byte = Convert.FromBase64String(strIV)

        'Dim key() As Byte = Encoding.UTF8.GetBytes(strKey)
        'Dim IV() As Byte = Encoding.UTF8.GetBytes(strIV)

        aes.Key = key
        aes.IV = IV

        Using ms As New MemoryStream
            ms.WriteByte(aes.Key.Length)
            ms.Write(aes.Key, 0, aes.Key.Length)
            ms.WriteByte(aes.IV.Length)
            ms.Write(aes.IV, 0, aes.IV.Length)

            Using cs As New CryptoStream(ms, aes.CreateEncryptor(), CryptoStreamMode.Write)
                Dim bytes() As Byte = Encoding.UTF8.GetBytes(strData)
                cs.Write(bytes, 0, bytes.Length)
            End Using

            'Return Convert.ToBase64String(ms.ToArray())
            Return (ms.ToArray())
        End Using
    Catch err As Exception
        MessageBox.Show("Error in AES_Encrypt function:" & err.Message)
    End Try
End Function

[cristoper]

I'm not familiar with .NET, but the only thing I don't follow is why are you writing the key and IV to the stream before the data?

ms.WriteByte(aes.Key.Length) ms.Write(aes.Key, 0, aes.Key.Length) ms.WriteByte(aes.IV.Length) ms.Write(aes.IV, 0, aes.IV.Length)

[nightcoding2021]

I'm not familiar enough with encryption to know exactly what's correct in that section. Possibly hence the problem. Hopefully someone with experience of doing this in .NET gets a chance to look.

[SouravPal95]

I am currently in the process of building an API that allows users to programmatically list products in their amazon seller accounts. I have been able to implement the OAuth process, signing mechanism for making api calls and have gone through the recommended steps provided in the use case guide for feeds API. The last step is the feeds API processing and report and mystery. For me, the feed processing status is DONE. After decrypting and decompressing the response, I get a JSON payload that looks like this. However, I am unable to understand the error, and how to rectify it based on the error code and messages displayed:

{
  "header": {
    "sellerId": "A1HGLY0OQBKE5U",
    "version": "2.0",
    "feedId": "50021018827"
  },
  "issues": [
    {
      "messageId": 1,
      "code": "4002008",
      "severity": "ERROR",
      "message": "The provided message did not meet the schema validation requirements for a feed message."
    },
    {
      "messageId": 2,
      "code": "4002008",
      "severity": "ERROR",
      "message": "The provided message did not meet the schema validation requirements for a feed message."
    },
    {
      "messageId": 3,
      "code": "4002008",
      "severity": "ERROR",
      "message": "The provided message did not meet the schema validation requirements for a feed message."
    },
    {
      "messageId": 4,
      "code": "4002008",
      "severity": "ERROR",
      "message": "The provided message did not meet the schema validation requirements for a feed message."
    },
    {
      "messageId": 5,
      "code": "4002008",
      "severity": "ERROR",
      "message": "The provided message did not meet the schema validation requirements for a feed message."
    },
    {
      "messageId": 6,
      "code": "4002008",
      "severity": "ERROR",
      "message": "The provided message did not meet the schema validation requirements for a feed message."
    },
    {
      "messageId": 7,
      "code": "4002008",
      "severity": "ERROR",
      "message": "The provided message did not meet the schema validation requirements for a feed message."
    },
    {
      "messageId": 8,
      "code": "4002008",
      "severity": "ERROR",
      "message": "The provided message did not meet the schema validation requirements for a feed message."
    },
    {
      "messageId": 9,
      "code": "4002008",
      "severity": "ERROR",
      "message": "The provided message did not meet the schema validation requirements for a feed message."
    },
    {
      "messageId": 10,
      "code": "4002008",
      "severity": "ERROR",
      "message": "The provided message did not meet the schema validation requirements for a feed message."
    }
  ],
  "summary": {
    "errors": 10,
    "warnings": 0,
    "messagesProcessed": 10,
    "messagesAccepted": 0,
    "messagesInvalid": 10
  }
}

The json payload that was originally uploaded was based on the JSON_LISTINGS_FEED feed type schema provided here

The exact payload before encryption looks like this:

{
  "header": {
    "sellerId": "A1HGLY0OQBKE5U",
    "version": "2.0"
  },
  "messages": [
    {
      "messageId": 1,
      "operationType": "UPDATE",
      "sku": "sk129"
    },
    {
      "messageId": 2,
      "operationType": "UPDATE",
      "sku": "sk1005"
    },
    {
      "messageId": 3,
      "operationType": "UPDATE",
      "sku": "sk131"
    },
    {
      "messageId": 4,
      "operationType": "UPDATE",
      "sku": "sk132"
    },
    {
      "messageId": 5,
      "operationType": "UPDATE",
      "sku": "sk133"
    },
    {
      "messageId": 6,
      "operationType": "UPDATE",
      "sku": "sk134"
    },
    {
      "messageId": 7,
      "operationType": "UPDATE",
      "sku": "sk135"
    },
    {
      "messageId": 8,
      "operationType": "UPDATE",
      "sku": "sk136"
    },
    {
      "messageId": 9,
      "operationType": "UPDATE",
      "sku": "sk137"
    },
    {
      "messageId": 10,
      "operationType": "UPDATE",
      "sku": "sk138"
    }
  ]
}

Any help will be deeply appreciated

[agb]

I am currently in the process of building an API that allows users to programmatically list products in their amazon seller accounts. I have been able to implement the OAuth process, signing mechanism for making api calls and have gone through the recommended steps provided in the use case guide for feeds API. The last step is the feeds API processing and report and mystery. For me, the feed processing status is DONE. After decrypting and decompressing the response, I get a JSON payload that looks like this. However, I am unable to understand the error, and how to rectify it based on the error code and messages displayed:

{
  "header": {
    "sellerId": "A1HGLY0OQBKE5U",
    "version": "2.0",
    "feedId": "50021018827"
  },
  "issues": [
    {
      "messageId": 1,
      "code": "4002008",
      "severity": "ERROR",
      "message": "The provided message did not meet the schema validation requirements for a feed message."
    },
    {
      "messageId": 2,
      "code": "4002008",
      "severity": "ERROR",
      "message": "The provided message did not meet the schema validation requirements for a feed message."
    },
    {
      "messageId": 3,
      "code": "4002008",
      "severity": "ERROR",
      "message": "The provided message did not meet the schema validation requirements for a feed message."
    },
    {
      "messageId": 4,
      "code": "4002008",
      "severity": "ERROR",
      "message": "The provided message did not meet the schema validation requirements for a feed message."
    },
    {
      "messageId": 5,
      "code": "4002008",
      "severity": "ERROR",
      "message": "The provided message did not meet the schema validation requirements for a feed message."
    },
    {
      "messageId": 6,
      "code": "4002008",
      "severity": "ERROR",
      "message": "The provided message did not meet the schema validation requirements for a feed message."
    },
    {
      "messageId": 7,
      "code": "4002008",
      "severity": "ERROR",
      "message": "The provided message did not meet the schema validation requirements for a feed message."
    },
    {
      "messageId": 8,
      "code": "4002008",
      "severity": "ERROR",
      "message": "The provided message did not meet the schema validation requirements for a feed message."
    },
    {
      "messageId": 9,
      "code": "4002008",
      "severity": "ERROR",
      "message": "The provided message did not meet the schema validation requirements for a feed message."
    },
    {
      "messageId": 10,
      "code": "4002008",
      "severity": "ERROR",
      "message": "The provided message did not meet the schema validation requirements for a feed message."
    }
  ],
  "summary": {
    "errors": 10,
    "warnings": 0,
    "messagesProcessed": 10,
    "messagesAccepted": 0,
    "messagesInvalid": 10
  }
}

The json payload that was originally uploaded was based on the JSON_LISTINGS_FEED feed type schema provided here

The exact payload before encryption looks like this:

{
  "header": {
    "sellerId": "A1HGLY0OQBKE5U",
    "version": "2.0"
  },
  "messages": [
    {
      "messageId": 1,
      "operationType": "UPDATE",
      "sku": "sk129"
    },
    {
      "messageId": 2,
      "operationType": "UPDATE",
      "sku": "sk1005"
    },
    {
      "messageId": 3,
      "operationType": "UPDATE",
      "sku": "sk131"
    },
    {
      "messageId": 4,
      "operationType": "UPDATE",
      "sku": "sk132"
    },
    {
      "messageId": 5,
      "operationType": "UPDATE",
      "sku": "sk133"
    },
    {
      "messageId": 6,
      "operationType": "UPDATE",
      "sku": "sk134"
    },
    {
      "messageId": 7,
      "operationType": "UPDATE",
      "sku": "sk135"
    },
    {
      "messageId": 8,
      "operationType": "UPDATE",
      "sku": "sk136"
    },
    {
      "messageId": 9,
      "operationType": "UPDATE",
      "sku": "sk137"
    },
    {
      "messageId": 10,
      "operationType": "UPDATE",
      "sku": "sk138"
    }
  ]
}

Any help will be deeply appreciated

How you can solved this problem?

[SouravPal95]

Essentially, Amazon is picky about which fields are needed for the feed submission to be valid. What I would recommend is to download the schema sample from the docs and paste it in an online schema validator. Then Paste your payload and check for what is missing or wrong....That is how I solved it...

[agb]

Essentially, Amazon is picky about which fields are needed for the feed submission to be valid. What I would recommend is to download the schema sample from the docs and paste it in an online schema validator. Then Paste your payload and check for what is missing or wrong....That is how I solved it...

Thank you for your reply. I couldn't solve it. Do you have an idea for this problem? https://github.com/amzn/selling-partner-api-models/issues/1968

[github-actions[bot]]

This is a very old issue that is probably not getting as much attention as it deserves. We encourage you to check if this is still an issue after the latest release and if you find that this is still a problem, please feel free to open a new issue and make a reference to this one.