andrewlansdowne
commented
1 year ago OK I think I've figured this out,
It's because the JSS image resizing whitelist blocks parsing of the querystring if ALL the querystring parameters from "protectedMediaQueryParameters" are not listed in the allowedMediaParams list (apart from rev which is ignored).
"extension" is added to the "protectedMediaQueryParameters" by Dianoga's webp config file and it seems to be required for the different images to be served/cached independently. So can't remove this.
Since it is an exact match of the combination of all the parameters, you can't realistically add all combinations of mw/mh/extension (perhaps you could if you only want webp since I think the comma used in extension=avif,webp would be an issue because allowedMediaParams uses commas as a separate)
I have therefore created a override of this class Sitecore.JavaScriptServices.Media.Pipelines.RestrictMediaRequest.ConfigurationWhitelistProcessor which ignores valid values of "extension" when checking against the resize whitelist. It still blocks if an arbitrary string is passed, which would enable the DOS attack vector the feature is intended to block.
public class ConfigurationWhitelistProcessor : Sitecore.JavaScriptServices.Media.Pipelines.RestrictMediaRequest.ConfigurationWhitelistProcessor
{
public ConfigurationWhitelistProcessor(BaseMediaManager mediaManager, WhitelistConfiguration whitelistConfiguration, BaseLog log, BaseSettings settings) : base(mediaManager, whitelistConfiguration, log, settings)
{
}
/// <summary>
/// Extension is added to the protectedMediaQueryParameters needed for each variation to be cached seperately.
/// But it should be ignored by the JSS image resize whitelist since it isn't listed in allowedMediaParams
/// as that would require every combination of mw/mh/extension to be added.
/// </summary>
/// <param name="queryString"></param>
/// <returns></returns>
protected override IDictionary<string, string> GetProtectedParams(NameValueCollection queryString)
{
var p = base.GetProtectedParams(queryString);
if (p.ContainsKey("extension") && IsValidExtension(p["extension"]))
{
p.Remove("extension");
}
return p;
}
string[] validExtensions = new string[] { "webp", "avif" };
private bool IsValidExtension(string extension)
{
if (!string.IsNullOrEmpty(extension))
{
var splitExt = extension.Split(',');
if (splitExt.All(s => validExtensions.Contains(s)))
{
return true;
}
}
return false;
}
}Ideally the list of valid extensions would come from the Dianoga enabled next-gen configs but I haven't figured out how to do that yet. The dianogaGetSupportedFormats isn't suitable as-is because you have to pass in the Accept header which defeats the point as I'd have to hardcode that so for now I'm just hardcoding the common next-gen extensions, that we are using.
This pipeline has to be replaced via config include:
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:role="http://www.sitecore.net/xmlconfig/role/" xmlns:set="http://www.sitecore.net/xmlconfig/set/">
<sitecore>
<pipelines>
<group groupName="javaScriptServices">
<pipelines>
<restrictMediaRequest>
<processor type="Sitecore.JavaScriptServices.Media.Pipelines.RestrictMediaRequest.ConfigurationWhitelistProcessor, Sitecore.JavaScriptServices.Media">
<patch:delete />
</processor>
<processor type="Your.Optimisation.Media.ConfigurationWhitelistProcessor, Your.Optimisation" resolve="true" />
</restrictMediaRequest>
</pipelines>
</group>
</pipelines>
</sitecore>
</configuration>
markgibbons25
kpicavet
AlexandruPaun4Ness
Version of Dianoga
6.1.0
Environment description
Sitecore 10.2 XP0 running locally with Headless Services 20.0.2 module installed, running a jss next.js headless site. Ensure you have some entries in the JSS media resizing whitelist (allowedMediaParams) e.g. mw=640.
What configs you have enabled
Dianoga.NextGenFormats.CDN.config Dianoga.Strategy.GetMediaStreamSync.config z.01.Dianoga.NextGenFormats.WebP.config z.02.Dianoga.NextGenFormats.Avif.config
Reproducible steps (1... 2... 3...) that cause the issue
What you expected to see, versus what you actually saw
I see an optimized resized jpeg, and in the Dianoga logs it says it optimized as a jpeg. I expect to see an optimized resized webp or avif file depending on the extension querystring.
Relevant logs
No errors, just the expected info logs saying the image was resized into jpeg. For example: 24172 00:22:37 INFO Dianoga: optimized /site/filename.jpg [original size: 1736039 bytes] [final size: 341112 bytes] [saved 1394927 bytes / 80.35%] [Optimized in 685ms] [Extension jpg]
Hi there, I've raised this issue to document the details of this issue as I am actively investigating and believe I have a fix for this which if it works I will submit a PR or at the least document it below. Thanks, Andy