CloudSearch: use POST for search request

JenteP commented 4 months ago

Checkboxes for prior research

[X] I've gone through Developer Guide and API reference
[X] I've checked AWS Forums and StackOverflow.
[X] I've searched for previous similar issues and didn't find any solution.

Describe the bug

We get an UnknownError (413) from the client when trying to search with a lot of clauses. Although it should still be below the 8kb limit specified in the documentation. In the aws v2 sdk there was a similar issue here in which the get request for the search domain was transformed to a post request. Because of this issue we were forced to downgrade the search client form the v3 sdk to v2.

SDK version number

@aws-sdk/client-cloudsearch-domain@3.496.0

Which JavaScript Runtime is this issue in?

Node.js

Details of the browser/Node.js/ReactNative version

v18.16.0

Reproduction Steps

const searchUsingV3Client = async () => {
    const params = {
        "query":"(and(or iban:'BE00000000000001' iban:'BE00000000000002' iban:'BE00000000000003' iban:'BE00000000000004' iban:'BE00000000000005' iban:'BE00000000000006' iban:'BE00000000000007' iban:'BE00000000000008' iban:'BE00000000000009' iban:'BE00000000000010' iban:'BE00000000000011' iban:'BE00000000000012' iban:'BE00000000000013' iban:'BE00000000000014' iban:'BE00000000000015' iban:'BE00000000000016' iban:'BE00000000000017' iban:'BE00000000000018' iban:'BE00000000000019' iban:'BE00000000000020' iban:'BE00000000000021' iban:'BE00000000000022' iban:'BE00000000000023' iban:'BE00000000000024' iban:'BE00000000000025' iban:'BE00000000000026' iban:'BE00000000000027' iban:'BE00000000000028' iban:'BE00000000000029' iban:'BE00000000000030' iban:'BE00000000000031' iban:'BE00000000000032' iban:'BE00000000000033' iban:'BE00000000000034' iban:'BE00000000000035' iban:'BE00000000000036' iban:'BE00000000000037' iban:'BE00000000000038' iban:'BE00000000000039' iban:'BE00000000000040' iban:'BE00000000000041' iban:'BE00000000000042' iban:'BE00000000000043' iban:'BE00000000000044' iban:'BE00000000000045' iban:'BE00000000000046' iban:'BE00000000000047' iban:'BE00000000000048' iban:'BE00000000000049' iban:'BE00000000000050' iban:'BE00000000000051' iban:'BE00000000000052' iban:'BE00000000000053' iban:'BE00000000000054' iban:'BE00000000000055' iban:'BE00000000000056' iban:'BE00000000000057' iban:'BE00000000000058' iban:'BE00000000000059' iban:'BE00000000000060' iban:'BE00000000000061' iban:'BE00000000000062' iban:'BE00000000000063' iban:'BE00000000000064' iban:'BE00000000000065' iban:'BE00000000000066' iban:'BE00000000000067' iban:'BE00000000000068' iban:'BE00000000000069' iban:'BE00000000000070' iban:'BE00000000000071' iban:'BE00000000000072' iban:'BE00000000000073' iban:'BE00000000000074' iban:'BE00000000000075' iban:'BE00000000000076' iban:'BE00000000000077' iban:'BE00000000000078' iban:'BE00000000000079' iban:'BE00000000000080' iban:'BE00000000000081' iban:'BE00000000000082' iban:'BE00000000000083' iban:'BE00000000000084' iban:'BE00000000000085' iban:'BE00000000000086' iban:'BE00000000000087' iban:'BE00000000000088' iban:'BE00000000000089' iban:'BE00000000000090' iban:'BE00000000000091' iban:'BE00000000000092' iban:'BE00000000000093' iban:'BE00000000000094' iban:'BE00000000000095' iban:'BE00000000000096' iban:'BE00000000000097' iban:'BE00000000000098' iban:'BE00000000000099' iban:'BE00000000000100' iban:'BE00000000000101' iban:'BE00000000000102' iban:'BE00000000000103' iban:'BE00000000000104' iban:'BE00000000000105' iban:'BE00000000000106' iban:'BE00000000000107' iban:'BE00000000000108' iban:'BE00000000000109' iban:'BE00000000000110' iban:'BE00000000000111' iban:'BE00000000000112' iban:'BE00000000000113' iban:'BE00000000000114' iban:'BE00000000000115' iban:'BE00000000000116' iban:'BE00000000000117' iban:'BE00000000000118' iban:'BE00000000000119' iban:'BE00000000000120' iban:'BE00000000000121' iban:'BE00000000000122' iban:'BE00000000000123' iban:'BE00000000000124' iban:'BE00000000000125' iban:'BE00000000000126' iban:'BE00000000000127' iban:'BE00000000000128' iban:'BE00000000000129' iban:'BE00000000000130' iban:'BE00000000000131' iban:'BE00000000000132' iban:'BE00000000000133' iban:'BE00000000000134' iban:'BE00000000000135' iban:'BE00000000000136' iban:'BE00000000000137' iban:'BE00000000000138' iban:'BE00000000000139' iban:'BE00000000000140' iban:'BE00000000000141' iban:'BE00000000000142' iban:'BE00000000000143' iban:'BE00000000000144' iban:'BE00000000000145' iban:'BE00000000000146' iban:'BE00000000000147' iban:'BE00000000000148' iban:'BE00000000000149' iban:'BE00000000000150' iban:'BE00000000000151' iban:'BE00000000000152' iban:'BE00000000000153' iban:'BE00000000000154' iban:'BE00000000000155' iban:'BE00000000000156' iban:'BE00000000000157' iban:'BE00000000000158' iban:'BE00000000000159' iban:'BE00000000000160' iban:'BE00000000000161' iban:'BE00000000000162' iban:'BE00000000000163' iban:'BE00000000000164' iban:'BE00000000000165' iban:'BE00000000000166' iban:'BE00000000000167' iban:'BE00000000000168' iban:'BE00000000000169' iban:'BE00000000000170' iban:'BE00000000000171' iban:'BE00000000000172' iban:'BE00000000000173' iban:'BE00000000000174' iban:'BE00000000000175' iban:'BE00000000000176' iban:'BE00000000000177' iban:'BE00000000000178' iban:'BE00000000000179' iban:'BE00000000000180' iban:'BE00000000000181' iban:'BE00000000000182' iban:'BE00000000000183' iban:'BE00000000000184' iban:'BE00000000000185' iban:'BE00000000000186' iban:'BE00000000000187' iban:'BE00000000000188' iban:'BE00000000000189' iban:'BE00000000000190' iban:'BE00000000000191' iban:'BE00000000000192' iban:'BE00000000000193' iban:'BE00000000000194' iban:'BE00000000000195' iban:'BE00000000000196' iban:'BE00000000000197' iban:'BE00000000000198' iban:'BE00000000000199' iban:'BE00000000000200'))",
        "queryParser": "structured",
        "size": 25,
        "sort": "iban desc",
        "start": 0
    }

    const cloudSearchDomainClient = new CloudSearchDomainClient({
        region: 'eu-central-1',
        endpoint: 'https://cloudsearch-domain.com',
    });
    console.log('Searching')
    try {
        const response = await cloudSearchDomainClient.send(new SearchCommand(
            params
        ));
    } catch (e) {
        console.log(`${e.message} ${JSON.stringify(e)}`)
    }
}

Observed Behavior

UnknownError {"name":"413","$fault":"client","$metadata":{"httpStatusCode":413,"attempts":1,"totalRetryDelay":0},"message":"UnknownError"}

Expected Behavior

Transform the get request to a post request on large queries

Possible Solution

See here

Additional Information/Context

No response

RanVaknin commented 4 months ago

Hi @JenteP ,

Thanks for reaching out and for providing the context. In the new SDKs we are moving away from hand written customizations since they are creating feature parity gaps (as seen here). The behavior of this operation working correctly with POST and not with GET points to a service side problem.

I have created an internal ticket with the OpenSearch service team to try and address this P117425632 and will move this to the cross SDK repo for tracking.

Thanks, Ran~

RanVaknin commented 4 months ago

aws / aws-sdk