Provide hooks

[JoyceBabu]

Is it possible to provide custom hooks when a file is optimized, so that we can use it to push optimized files to CDN? The support can be as simple as a script file which is invoked whenever a file is optimized, or a full system that allows modifying the optimized script with lua.

[jmarantz]

I thought we had a longstanding feature-request open against supporting Push CDNs, but I can't find it now.

Let's anoint this as that feature request :)

Note that for this to work, we'd have to consider the opimization incomplete until the push had completed, which might take some time. E.g. I can imagine some CDNs where the full propagation to all replicas might take minutes.

[JoyceBabu]

Are you referring to one of these discussions on Google groups?

• https://groups.google.com/forum/#!searchin/mod-pagespeed-discuss/Cdn%7Csort:relevance/mod-pagespeed-discuss/GroLYaYru54/HjkeenhseQ4J
• https://groups.google.com/forum/#!searchin/mod-pagespeed-discuss/Cdn%7Csort:relevance/mod-pagespeed-discuss/ZWYukDZWvbY/WzIzzvUPVToJ

[JoyceBabu]

Note that for this to work, we'd have to consider the opimization incomplete until the push had completed, which might take some time. E.g. I can imagine some CDNs where the full propagation to all replicas might take minutes.

I think page speed should not concern itself with pushing to the CDN. Instead it should provide limited access to the optimized resources. It would allow others to add further customizations as per their requirements, including support for their favourite CDN.

By limited access, I am referring to

• Callbacks when
  • a new unoptimized resource is detected
  • a resource is optimized
  • a resource expires
• access / modify headers of the optimized resource
• API to force expire a resource
• API to mark/unmark an optimized resource as unavailable

Since lua scripting is available for apache and nginx, it might be a good option.

[jeffkaufman]

A workaround for a "callback on optimized resource creation" would be to use a cache that supports subscribing to notifications, and then notice when new optimized resources are written to cache. Ways to get a notification stream from the cache:

• put the file cache on a tmpfs mount and use fanotify
• use inotify to, with somewhat more work, watch an existing file system cache directory that isn't a whole partition.
• use our redis integration and then set up redis notifications

You would want to watch for new cache keys matching v3/yoursite/.../*.pagespeed.*.

The main downside of this workaround is it ties you pretty tightly to mod_pagespeed's internals. While we don't have any plans to change these, and haven't in a long time, this wouldn't be a supported use and a new release might move things around in a way that broke your implementation.

[jeffkaufman]

In terms of adding a new feature to mod_pagespeed, I think the best way for this to work is to be able to configure mod_pagespeed to send http requests to some endpoint when those events happen. For example, if http://modpagespeed.com/rewrite_javascript.js.pagespeed.jm.1o978_K0_L.js was just created, then the url could be in the post body. If something wanted the optimized resource contents, it could follow up on the ping by GETting the given URL.

(I believe "API to force expire a resource" already exists with PURGE handling.)

[JoyceBabu]

Thank you for suggesting the work around. I will try it.

The ability to modify the optimized resource and/or it's meta data would be a great feature, as it allows the creation of plugins to add additional features to page speed that are not popular/important enough to be added to the core.

On Jan 25, 2017 11:51 PM, "Jeff Kaufman" notifications@github.com wrote:

In terms of adding a new feature to mod_pagespeed, I think the best way for this to work is to be able to configure mod_pagespeed to send http requests to some endpoint when those events happen. For example, if http://modpagespeed.com/rewrite_javascript.js.pagespeed.jm.1o978_K0_L.js was just created, then the url could be in the post body. If something wanted the optimized resource contents, it could follow up on the ping by GETting the given URL.

(I believe "API to force expire a resource" already exists with PURGE handling https://modpagespeed.com/doc/system#purge_cache.)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/pagespeed/mod_pagespeed/issues/1487#issuecomment-275189375, or mute the thread https://github.com/notifications/unsubscribe-auth/AAesnkO3N6Ar1LJCKAq3XIcpTHs0ES1Uks5rV5K1gaJpZM4LtcAV .