Closed xtuc closed 1 year ago
what about X-Robots-Tag for example where rules specified without a user agent are valid for all
console.log( new Headers([ ['x-robots-tag', 'noindex'], ['x-robots-tag', 'unavailable_after: 25 Jun 2099 15:00:00'], ['x-robots-tag', 'somebot: nofollow'], ['x-robots-tag', 'otherbot: noindex, nofollow'], ['x-robots-tag', 'noarchive'], ]).get('x-robots-tag') == 'noindex, unavailable_after: 25 Jun 2099 15:00:00, somebot: nofollow, otherbot: noindex, nofollow, noarchive' )
You always have to put the *bot
last in your array.
i.e. noindex, noarchive
should be the first 2 in your output
It doesn't fix , nofollow
though.
. i.e.
noindex, noarchive
should be the first 2 in your output
they "should" but what to do when a server sends them after other ua rules?
Since https://github.com/whatwg/fetch/commit/42464c8c3d2fd3437a19fc6afd2438a0fd42dde8
Headers.prototype.getAll
has been deprecated/removed from the spec and implementations.I understand that in browsers (including service workers) filtered responses don't include headers that could benefit from the getAll method. However, some JavaScript environment doesn't need to filter response/request headers, like serverless plateforms (for instance Cloudflare Workers) or maybe Nodejs at some point.
For example editing Cookie headers:
Spliting the combined header value by
,
will give an invalid result. InsteadgetAll
could be used to retrieve the individual headers.