Closed blowsie closed 7 years ago
Good question; I'm wondering the same thing, especially you can use Apache with prerender with no separate app, just a few lines of config in the .htaccess file:
Concept
https://gist.github.com/blowsie/04c183c62a432f6450c9
<?xml version="1.0"?>
<configuration>
<system.webServer>
<httpProtocol>
<customHeaders>
<add name="X-Prerender-Token" value="XXXXXXXXXXXXX" />
</customHeaders>
</httpProtocol>
<rewrite>
<rules>
<!--# Only proxy the request to Prerender if it's a request for HTML-->
<rule name="Prerender" stopProcessing="true">
<match url="^(?!.*?(\.js|\.css|\.xml|\.less|\.png|\.jpg|\.jpeg|\.gif|\.pdf|\.doc|\.txt|\.ico|\.rss|\.zip|\.mp3|\.rar|\.exe|\.wmv|\.doc|\.avi|\.ppt|\.mpg|\.mpeg|\.tif|\.wav|\.mov|\.psd|\.ai|\.xls|\.mp4|\.m4a|\.swf|\.dat|\.dmg|\.iso|\.flv|\.m4v|\.torrent))(.*)" ignoreCase="false" />
<conditions logicalGrouping="MatchAny">
<add input="{HTTP_USER_AGENT}" pattern="baiduspider|facebookexternalhit|twitterbot" />
<add input="{QUERY_STRING}" pattern="_escaped_fragment_" ignoreCase="false" />
</conditions>
<action type="Rewrite" url="http://service.prerender.io/http://yourwebsite.com/{R:2}" />
</rule>
</rules>
</rewrite>
</system.webServer>
</configuration>
Really interesting idea - with regards to that original concept though, I believe that will set the X-Prerender-Token on all requests - this should be kept private and only between you and the prerender service.
Off the top of my head, I wonder if you can do something like follows with outbound rules? The general idea is to match any request going to the service.prerender.io site and add on the X header in that instance only, rather than declaring it on the customHeaders section?
<outboundRules>
<rule name="Set prerender header">
<match serverVariable="RESPONSE_X_Prerender_Token" pattern=".*" />
<conditions>
<add input="{REQUEST_URI}" pattern="^http://service\.prerender\.io" />
</conditions>
<action type="Rewrite" value="XXXXXXXXXXXXXXXX"/>
</rule>
</outboundRules>
Unfortunately I'm not going to have a chance to have a play until later this week/weekend, but just in case anyone else does - maybe you can use the above?
From my investigations, you cannot rewrite to an absolute url outside of the current site without installing ARR and configuring a reverse proxy on the server. While this approach might work, the overhead of ARR+Rewrite will probably be a lot larger than this custom module.
EDIT: I have taken some time to build a URL Rewrite solution. While it might come with a little bit of overhead, most static sites might not want to load some additional C# code just for this. This solution enables the use of prerender.io through configuration files only.
<rule name="Proxy to prerender.io for crawlers" enabled="false" stopProcessing="true">
<match url="^(?!.*?(\.js|\.css|\.xml|\.less|\.png|\.jpg|\.jpeg|\.gif|\.pdf|\.doc|\.txt|\.ico|\.rss|\.zip|\.mp3|\.rar|\.exe|\.wmv|\.doc|\.avi|\.ppt|\.mpg|\.mpeg|\.tif|\.wav|\.mov|\.psd|\.ai|\.xls|\.mp4|\.m4a|\.swf|\.dat|\.dmg|\.iso|\.flv|\.m4v|\.torrent))(.*)" ignoreCase="false" />
<conditions logicalGrouping="MatchAny">
<add input="{HTTP_USER_AGENT}" pattern="bingbot|baiduspider|facebookexternalhit|twitterbot|yandex|rogerbot|linkedinbot|embedly|bufferbot|quora link preview|showyoubot|outbrain" />
<add input="{QUERY_STRING}" pattern="_escaped_fragment_" ignoreCase="false" />
</conditions>
<serverVariables>
<!-- Needs to be present in the Rewrite Module's authorized server variable list -->
<set name="HTTP_X_PRERENDER_TOKEN" value="_ENTER_TOKEN_HERE_"/>
</serverVariables>
<!-- External rewrites requires ARR to be installed with the proxy service enabled on the server -->
<action type="Rewrite" url="http://service.prerender.io/{MapProtocol:{HTTPS}}://_ENTER_BASE_URL_HERE_/{R:0}" />
</rule>
And the associated rewrite map:
<rewriteMap name="MapProtocol">
<add key="on" value="https" />
<add key="off" value="http" />
</rewriteMap>
I am not able to get this working . Can someone help ? I am getting a 500 internal server error when trying the same
please check here. www.vbuy.in/product/samsung-galaxy-grand-2-black-/1975?_escapedfragment
http://www.vbuy.in/product/samsung-galaxy-grand-2-black-/1975?_escaped_fragment_=
is redirecting to:
http://service.prerender.io/http://www.vbuy.in/product/samsung-galaxy-grand-2-black-/1975?_escaped_fragment_=
which is incorrect. Something isn't set up right there.
It should proxy behind the scenes to our service without doing a redirect. Send me the config set up that you have at todd@prerender.io and I'll look into why that's happening.
Todd, we will discuss in mail. Thanks for your continuous support in mail.
-Siva.
Any update? Can i close this ?
I think so. Thanks for your help @greengerong !!
Just a thought is there a reason this isn't using the IIS re-write module? Did you try and develop using this? Or is there any limitations or reasons this wont work?
http://www.iis.net/downloads/microsoft/url-rewrite