Accept brotli encoding

[rudolf]

Summary

Adds "br" to the Accept-encoding header. This will cause performance tests to use brotli compression if Kibana supports it.

Checklist

Delete any items that are not applicable to this PR.

• [ ] Branch is successfully tested on Kibana CI
• [ ] Unit tests are added

[rudolf]

@dmlemeshko I've ran a build with this branch and brotli support on Kibana https://kibana-ci.elastic.co/job/elastic+kibana+load-testing/2159/parameters/

The numbers are different but still not 100% sure how to verify that brotli compression is actually used for all our API requests.

[dmlemeshko]

The numbers are different but still not 100% sure how to verify that brotli compression is actually used for all our API requests.

I will try to run locally with small load and enable verbose logging, it might help. I will add details as soon as I got it.

[dmlemeshko]

Hi @rudolf , sorry for late reply. I can see Gatling using the following header while making all the requests: accept-encoding: gzip, deflate, br

HTTP request:
POST http://localhost:5620/internal/security/login
headers:
    kbn-xsrf: xsrf
    accept-language: en-GB,en-US;q=0.9,en;q=0.8
    accept: application/json
    user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36
    Content-Type: application/json
    accept-encoding: gzip, deflate, br
    host: localhost:5620
    content-length: 144
body:StringChunksRequestBody{charset=UTF-8, content={"providerType":"basic","providerName":"basic","currentURL":"http://localhost:5620/login","params":{"username":"elastic","password":"changeme"}}}
=========================
HTTP response:
status:
    200 OK
headers:
    x-content-type-options: nosniff
    referrer-policy: no-referrer-when-downgrade
    kbn-name: Dzmitrys-MBP-2
    kbn-license-sig: bda6ce9048cf8d95e848e0acb501adc11d4a99d4840b0f523352b094d85b8f4d
    content-type: application/json; charset=utf-8
    cache-control: private, no-cache, no-store, must-revalidate
    set-cookie: sid=Fe26.2**db709a44de56247146f8f63d478c3912d124733faa0addafccbec66f0aed42ed*r1RyWBX3HQ09RgT-yT9Sbg*oEXZ-ijVtvrISaw1Fy5or7F-pSak0u4p4CxXuFDOhAMS90XRy7FJ1f_VFM35oUdFHcitIxD3C9E31hsd_BAtkHsJRlExYjI2zDekRE4l6vGyIHtXNSfPDZn0PqbX8IhRDkNqXJ5Cw-vHItK14Hs0ORavxxB3sb_Gc_gsBS6_bsB3YjY1XNxdmauGLF8WxpKsaoN84xinldy56pPDsoICYWBT5xucNx3XUHlphpFEgsGd9Rvjx-PE_jm-GIpDKpqR**5c375c8bff8ed368251c027f4e3ca439d661a7f219a236a07ab318ffd67a7f6d*ev_pFxPAac539sHTgXCxRU5o-tr4WSCESjASfu1-lE8; HttpOnly; Path=/
    Date: Wed, 12 Oct 2022 14:47:40 GMT
    Connection: keep-alive
    Keep-Alive: timeout=120
    content-length: 16

body:
{"location":"/"}

I'm not sure if that confirms that Kibana is using brotli for the all requests, probably we can add some logging on Kibana side?

[rudolf]

Gattling's request looks correct but I'm surprised that Kibana is responding without any Content-encoding header. main doesn't support brotli but it should gzip this response. I'll try to investigate this.

[rudolf]

Oh the reason is that hapi doesn't compress responses less than 1024 bytes so it makes sense that this response wasn't compressed.

[dmlemeshko]

LGTM

[dmlemeshko]

Gattling's request looks correct but I'm surprised that Kibana is responding without any Content-encoding header. main doesn't support brotli but it should gzip this response

I went again through the logs and found this example with content-encoding: gzip. I guess we expect to see content-encoding:br here?

HTTP request:
POST http://localhost:5620/api/saved_objects/_bulk_resolve
headers:
    kbn-version: 8.6.0
    Referer: http://localhost:5620/app/lens
    Sec-Fetch-Site: same-origin
    Connection: keep-alive
    Sec-Fetch-Mode: cors
    accept-language: en-GB,en-US;q=0.9,en;q=0.8
    accept: */*
    user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36
    Sec-Fetch-Dest: empty
    Origin: http://localhost:5620
    Content-Type: application/json
    accept-encoding: gzip, deflate, br
    cookie: sid=Fe26.2**06eba37abd40c289e2f37dfb8b8124e04034bc9eab49e734fccd675d6cf35e2d*BnAf3ASWsjTcgcDA32fLkQ*dKrMVG0hvleQmiyVx2PxUPE61o5WUpJEfmWEw0tfwqil1AQU7bo4wJ-cA8JkA-QsSVPyRHLZDVEfYKH9imQLedkqYWgzVafSb9-yYfPxOu5EhoP6_AMrGObbiJn1BLLHd81nOngfubPTkQg-paRqS8X9eqO1RKFIFwt4o5H7L9GjGV5W5pB7GUqbLLh-mv4s3V5FvH2iKDMDCWGbtUCrAvokIdWmHHVQBUoHJwUnIBuy1kPwVNhf8lHieb2DjnIm**119688380bd36e0f30e1cea5fb683809139bfcaa50dd2352056ff049653e1324*BxAE7j-RnDUsr3y9VEmtB8n2ZnSiBqnlEOoWOTsViMI
    host: localhost:5620
    content-length: 61
cookies:
    sid=Fe26.2**06eba37abd40c289e2f37dfb8b8124e04034bc9eab49e734fccd675d6cf35e2d*BnAf3ASWsjTcgcDA32fLkQ*dKrMVG0hvleQmiyVx2PxUPE61o5WUpJEfmWEw0tfwqil1AQU7bo4wJ-cA8JkA-QsSVPyRHLZDVEfYKH9imQLedkqYWgzVafSb9-yYfPxOu5EhoP6_AMrGObbiJn1BLLHd81nOngfubPTkQg-paRqS8X9eqO1RKFIFwt4o5H7L9GjGV5W5pB7GUqbLLh-mv4s3V5FvH2iKDMDCWGbtUCrAvokIdWmHHVQBUoHJwUnIBuy1kPwVNhf8lHieb2DjnIm**119688380bd36e0f30e1cea5fb683809139bfcaa50dd2352056ff049653e1324*BxAE7j-RnDUsr3y9VEmtB8n2ZnSiBqnlEOoWOTsViMI, path=/, HTTPOnly
body:StringChunksRequestBody{charset=UTF-8, content=[{"id":"c762b7a0-f5ea-11eb-a78e-83aac3c38a60","type":"lens"}]}
=========================
HTTP response:
status:
    200 OK
headers:
    x-content-type-options: nosniff
    referrer-policy: no-referrer-when-downgrade
    kbn-name: Dzmitrys-MBP-2
    kbn-license-sig: 0323531b1c8a72801a345857644d9d2ad52f0ed62acfb99d3e483106f36b3896
    content-type: application/json; charset=utf-8
    cache-control: private, no-cache, no-store, must-revalidate
    set-cookie: sid=Fe26.2**c19c02fc3a8572efebad492116b7747a3b1cd57c7b350e583f4df12b09b858e6*QY3eMq_U9Bz4A3YzEZi40Q*iUo6ro1zv8ZKVUSaL-lQkM-uetzvXaij00otGqUnvmlFhPJox9WwfC23hCbUsLGlFHcK_2xX8p_XFyMlWlIxrzpgWmYcdLv-eGYZcYDi_zn9-iHbTne-cgQCzQXaLlxnqeJ3ZPSb6uZGmHoYo0kFLrc47WC33kBxibsYl8O81qzOKFcfcmRJNMq1iLeykq5kA92mOEkbZITn_ECw2DC-abrFpWzTVNQF38beavS9_lxhDKVa8vrWMh3j5aXVjpEj**3f008d4e5bd1f43fad208259f13e65dddba0e2252ecde9e08aa4a0d3c9e1f304*cl1OYtFzqQtV14WteZfSDukKVytec8b87Vi0tWM6_vc; HttpOnly; Path=/
    vary: accept-encoding
    Date: Wed, 12 Oct 2022 14:37:36 GMT
    Connection: keep-alive
    Keep-Alive: timeout=120
    Transfer-Encoding: chunked
    content-encoding: gzip

body:
{"resolved_objects":[{"saved_object":{"id":"c762b7a0-f5ea-11eb-a78e-83aac3c38a60","type":"lens","namespaces":["default"],"updated_at":"2022-10-12T14:36:43.780Z","version":"WzE1MCwxXQ==","attributes":{"state":{"datasourceStates":{"formBased":{"layers":{"c7478794-6767-4286-9d65-1c0ecd909dd8":{"columnOrder":["8289349e-6d1b-4abf-b164-0208183d2c34","041db33b-5c9c-47f3-a5d3-ef5e255d1663","041db33b-5c9c-47f3-a5d3-ef5e255d1663X0","041db33b-5c9c-47f3-a5d3-ef5e255d1663X1"],"columns":{"041db33b-5c9c-47f3-a5d3-ef5e255d1663":{"customLabel":true,"dataType":"number","isBucketed":false,"label":"% of target ($10k)","operationType":"formula","params":{"format":{"id":"percent","params":{"decimals":0}},"formula":"sum(taxful_total_price) / 10000 - 1","isFormulaBroken":false},"references":["041db33b-5c9c-47f3-a5d3-ef5e255d1663X1"],"scale":"ratio"},"041db33b-5c9c-47f3-a5d3-ef5e255d1663X0":{"customLabel":true,"dataType":"number","isBucketed":false,"label":"Part of Weekly revenue","operationType":"sum","scale":"ratio","sourceField":"taxful_total_price"},"041db33b-5c9c-47f3-a5d3-ef5e255d1663X1":{"customLabel":true,"dataType":"number","isBucketed":false,"label":"Part of Weekly revenue","operationType":"math","params":{"tinymathAst":{"args":[{"args":["041db33b-5c9c-47f3-a5d3-ef5e255d1663X0",10000],"location":{"max":32,"min":0},"name":"divide","text":"sum(taxful_total_price) / 10000 ","type":"function"},1],"location":{"max":35,"min":0},"name":"subtract","text":"sum(taxful_total_price) / 10000 - 1","type":"function"}},"references":["041db33b-5c9c-47f3-a5d3-ef5e255d1663X0"],"scale":"ratio"},"8289349e-6d1b-4abf-b164-0208183d2c34":{"dataType":"date","isBucketed":true,"label":"order_date","operationType":"date_histogram","params":{"interval":"1d","includeEmptyRows":true},"scale":"interval","sourceField":"order_date"}},"incompleteColumns":{}}}}},"filters":[],"query":{"language":"kuery","query":""},"visualization":{"axisTitlesVisibilitySettings":{"x":false,"yLeft":false,"yRight":true},"fittingFunction":"None","gridlinesVisibilitySettings":{"x":true,"yLeft":true,"yRight":true},"layers":[{"accessors":["041db33b-5c9c-47f3-a5d3-ef5e255d1663"],"layerId":"c7478794-6767-4286-9d65-1c0ecd909dd8","seriesType":"bar_stacked","xAccessor":"8289349e-6d1b-4abf-b164-0208183d2c34","layerType":"data"}],"legend":{"isVisible":true,"position":"right","legendSize":"auto"},"preferredSeriesType":"bar_stacked","tickLabelsVisibilitySettings":{"x":true,"yLeft":true,"yRight":true},"valueLabels":"hide","yLeftExtent":{"mode":"full"},"yRightExtent":{"mode":"full"}}},"title":"% of target revenue ($10k)","visualizationType":"lnsXY"},"references":[{"id":"ff959d40-b880-11e8-a6d9-e546fe2bba5f","name":"indexpattern-datasource-current-indexpattern","type":"index-pattern"},{"id":"ff959d40-b880-11e8-a6d9-e546fe2bba5f","name":"indexpattern-datasource-layer-c7478794-6767-4286-9d65-1c0ecd909dd8","type":"index-pattern"}],"migrationVersion":{"lens":"8.6.0"},"coreMigrationVersion":"8.6.0"},"outcome":"exactMatch"}]}

[rudolf]

I guess we expect to see content-encoding:br here?

Yes, but not on main yet as https://github.com/elastic/kibana/pull/142334 hasn't been merged yet.

[dmlemeshko]

Yes, but not on main yet as https://github.com/elastic/kibana/pull/142334 hasn't been merged yet.

Did another test with rockdaboot/use-brotli-compression branch and can see content-encoding: br header in response :tada: