URL-Rewrite

[blowsie]

Just a thought is there a reason this isn't using the IIS re-write module? Did you try and develop using this? Or is there any limitations or reasons this wont work?

http://www.iis.net/downloads/microsoft/url-rewrite

[jaydge]

Good question; I'm wondering the same thing, especially you can use Apache with prerender with no separate app, just a few lines of config in the .htaccess file:

https://gist.github.com/thoop/8072354

[blowsie]

Concept

https://gist.github.com/blowsie/04c183c62a432f6450c9

<?xml version="1.0"?>
<configuration>
    <system.webServer>
        <httpProtocol>
            <customHeaders>
                <add name="X-Prerender-Token" value="XXXXXXXXXXXXX" />
            </customHeaders>
        </httpProtocol>
        <rewrite>
            <rules>
                <!--# Only proxy the request to Prerender if it's a request for HTML-->
                <rule name="Prerender" stopProcessing="true">
                    <match url="^(?!.*?(\.js|\.css|\.xml|\.less|\.png|\.jpg|\.jpeg|\.gif|\.pdf|\.doc|\.txt|\.ico|\.rss|\.zip|\.mp3|\.rar|\.exe|\.wmv|\.doc|\.avi|\.ppt|\.mpg|\.mpeg|\.tif|\.wav|\.mov|\.psd|\.ai|\.xls|\.mp4|\.m4a|\.swf|\.dat|\.dmg|\.iso|\.flv|\.m4v|\.torrent))(.*)" ignoreCase="false" />
                    <conditions logicalGrouping="MatchAny">
                        <add input="{HTTP_USER_AGENT}" pattern="baiduspider|facebookexternalhit|twitterbot" />
                        <add input="{QUERY_STRING}" pattern="_escaped_fragment_" ignoreCase="false" />
                    </conditions>
                    <action type="Rewrite" url="http://service.prerender.io/http://yourwebsite.com/{R:2}" />
                </rule>
            </rules>
        </rewrite>
    </system.webServer>
</configuration>

[mgoodfellow]

Really interesting idea - with regards to that original concept though, I believe that will set the X-Prerender-Token on all requests - this should be kept private and only between you and the prerender service.

Off the top of my head, I wonder if you can do something like follows with outbound rules? The general idea is to match any request going to the service.prerender.io site and add on the X header in that instance only, rather than declaring it on the customHeaders section?

     <outboundRules>
        <rule name="Set prerender header">
          <match serverVariable="RESPONSE_X_Prerender_Token" pattern=".*" />
          <conditions>
            <add input="{REQUEST_URI}" pattern="^http://service\.prerender\.io" />
          </conditions>
          <action type="Rewrite" value="XXXXXXXXXXXXXXXX"/>
        </rule>
      </outboundRules>

Unfortunately I'm not going to have a chance to have a play until later this week/weekend, but just in case anyone else does - maybe you can use the above?

[kawazoe]

From my investigations, you cannot rewrite to an absolute url outside of the current site without installing ARR and configuring a reverse proxy on the server. While this approach might work, the overhead of ARR+Rewrite will probably be a lot larger than this custom module.

EDIT: I have taken some time to build a URL Rewrite solution. While it might come with a little bit of overhead, most static sites might not want to load some additional C# code just for this. This solution enables the use of prerender.io through configuration files only.

<rule name="Proxy to prerender.io for crawlers" enabled="false" stopProcessing="true">
    <match url="^(?!.*?(\.js|\.css|\.xml|\.less|\.png|\.jpg|\.jpeg|\.gif|\.pdf|\.doc|\.txt|\.ico|\.rss|\.zip|\.mp3|\.rar|\.exe|\.wmv|\.doc|\.avi|\.ppt|\.mpg|\.mpeg|\.tif|\.wav|\.mov|\.psd|\.ai|\.xls|\.mp4|\.m4a|\.swf|\.dat|\.dmg|\.iso|\.flv|\.m4v|\.torrent))(.*)" ignoreCase="false" />
    <conditions logicalGrouping="MatchAny">
        <add input="{HTTP_USER_AGENT}" pattern="bingbot|baiduspider|facebookexternalhit|twitterbot|yandex|rogerbot|linkedinbot|embedly|bufferbot|quora link preview|showyoubot|outbrain" />
        <add input="{QUERY_STRING}" pattern="_escaped_fragment_" ignoreCase="false" />
    </conditions>
    <serverVariables>
    <!-- Needs to be present in the Rewrite Module's authorized server variable list -->
    <set name="HTTP_X_PRERENDER_TOKEN" value="_ENTER_TOKEN_HERE_"/>
    </serverVariables>
    <!-- External rewrites requires ARR to be installed with the proxy service enabled on the server -->
    <action type="Rewrite" url="http://service.prerender.io/{MapProtocol:{HTTPS}}://_ENTER_BASE_URL_HERE_/{R:0}" />
</rule>

And the associated rewrite map:

<rewriteMap name="MapProtocol">
    <add key="on" value="https" />
    <add key="off" value="http" />
</rewriteMap>

[sivastyle]

I am not able to get this working . Can someone help ? I am getting a 500 internal server error when trying the same

please check here. www.vbuy.in/product/samsung-galaxy-grand-2-black-/1975?_escapedfragment

[thoop]

http://www.vbuy.in/product/samsung-galaxy-grand-2-black-/1975?_escaped_fragment_= is redirecting to: http://service.prerender.io/http://www.vbuy.in/product/samsung-galaxy-grand-2-black-/1975?_escaped_fragment_= which is incorrect. Something isn't set up right there.

It should proxy behind the scenes to our service without doing a redirect. Send me the config set up that you have at todd@prerender.io and I'll look into why that's happening.

[sivastyle]

Todd, we will discuss in mail. Thanks for your continuous support in mail.

-Siva.

[greengerong]

Any update? Can i close this ?

[thoop]

I think so. Thanks for your help @greengerong !!