cache vary problem with normalized_ae ( always prefer gzip ) ？

[esmq2092]

I had set proxy.config.http.normalize_ae to 2 and enabled brotli ( br ) compression，

when ats(v9.0.0) had cached both gzip and brotli compressed version of the object,
and if gzip object was generated after brotli one， ats will always return gzip when client supports both gzip and brotli.

steps to reproduce the problem:

1、Accept-Encoding: gzip,br ats will return br encoding as expected

2、Accept-Encoding: gzip ats will return gzip encoding as expected

3、Accept-Encoding: gzip,br ** _ats will always return gzip encoding , which should be br encoding._ **

4、Accept-Encoding: br ats will return br encoding as expected.

[esmq2092]

i found that normalized_ae happend in "READ_REQUEST_HDR" which cause the problem.

[bryancall]

@esmq2092 are you going to work on this?

[esmq2092]

i have found a workaround: using header_rewrite

cond %{READ_REQUEST_HDR_HOOK} [AND] cond %{HEADER:Accept-Encoding} /br/ set-header Accept-Encoding br

[bryancall]

Here is a link to the docs: https://docs.trafficserver.apache.org/admin-guide/files/records.config.en.html?proxy-config-http-normalize-ae

[rob05c]

normalize_ae affects what ATS sends the upstream parent or origin. It doesn't affect what ATS sends the client.

For example, if a client sends the header Accept-Encoding: gzip,br,deflate,foo,bar,baz and normalize_ae is 1 (gzip), ATS then sends the upstream parent/origin Accept-Encoding: gzip. This in no way affects what ATS sends the client.

Whatever the parent responds with, ATS will cache, with the response's Content-Encoding as variant.

When ATS gets a new client request, it will check the cache for acceptable variants from the original client request (not the normalized value). ATS should respect any client quality values (https://datatracker.ietf.org/doc/html/rfc7231#section-5.3.1). But if qualities are absent or equal, ATS may respond with any valid cached value.

If I understand correctly, you seem to expect that normalize_ae means "prefer this value when sending to the client." It doesn't. It just means "modify the header ATS sends upstream." Nothing more. The part of ATS that determines if a cached object exists doesn't know anything about the normalize_ae setting.

If all quality values are equal, the cached variant that ATS returns will be completely arbitrary. The normalize_ae setting doesn't affect that in any way.

We could theoretically add an ATS setting to "prefer encoding." But no setting like that exists today.

If you need to alter the quality, the right solution is to either make your client send an Accept-Encoding with quality values preferring brotli (if you control the client), or a Header Rewrite like you did.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. Marking it stale to flag it for further consideration by the community.