Add support for Googlebot, Bingbot, and Yandex

[thoop]

Thanks for making this repo. I've had a few people use it so far and say it works great!

One thing I noticed is that the URL isn't checked for _escaped_fragment_ so the crawlers that currently support it (Googlebot for now, Bingbot, and Yandex) are not served prerendered pages.

That should be a pretty easy check to add just before here.

Either that, or Googlebot, bingbot, and Yandex user agents should be added to the default list of user agents being checked. John Mueller has stated that as long as your prerendered content you serve back to Googlebot based on a user agent check is identical to the page your users see, you won't be penalized for cloaking in that case: https://groups.google.com/d/msg/js-sites-wg/CkLSG3TxxK0/LSJtiDxNCwAJ

[jinty]

Thanks for the feedback. Let me see if I understand it correctly, I just need to prerender if _escapedfragment is in the querystring?

I've made a pull request to that effect. I'd appreciate it if you give it a once-over: https://github.com/jinty/prerender-cloudfront/pull/6

[thoop]

Yes, that looks correct as long as querystring is a string on the lambda request and not an object.

Thanks for adding that! 😄

[jinty]

merged, thanks for bringing it up

[thoop]

Thanks!!