May 31, 2019

IIS oddity of the week

Written by Andy van Stokkum. 2 min read

The other day while trying to configure an IIS rewrite rule I came across some pretty odd IIS behavior. In an aim to improve the organic search results of a Single Page App, I needed to forward web requests from search engines to https://prerender.io.

In order for prerender.io to recognise the request as legitimate and render the page in headless-chrome, an HTTP request header X-Prerender-Token needs to be added. Now you would logically think the following syntax would be sufficient:

<rule name="Prerender" enabled="true" stopProcessing="true">
    <serverVariables>
        <set name="X-Prerender-Token" value="{my-secret-token}" />
    </serverVariables>
    <match url="^(?!.*?(\.js|\.css|\.xml|\.png|\.jpg|\.jpeg|\.gif|\.pdf|\.doc|\.txt|\.ico))(.*)" ignoreCase="true" />
    <conditions logicalGrouping="MatchAny">
        <add input="{HTTP_USER_AGENT}" pattern="bingbot|googlebot|baiduspider|twitterbot|facebookexternalhit|rogerbot|linkedinbot|embedly|quora link preview|showyoubot|outbrain|pinterest|slackbot|vkShare|W3C_Validator" />
        <add input="{QUERY_STRING}" pattern="(.*)_escaped_fragment_(.*)" ignoreCase="false" />
    </conditions>
    <action type="Rewrite" url="https://static-ssl.prerender.io/https://www.my-spa-site{REQUEST_URI}" appendQueryString="false" />
</rule>

But it turns out this fails.

Luckily via a Stack Overflow post I came across this  Microsoft document:

Note About Request Headers The request headers are set by using the same mechanism as for server variables, but with a special naming convention. If a server variable name in the <serverVariables> collection starts with “HTTP_” then this results in an HTTP request header being set in accordance to the following naming convention:

  • All underscore ("_") symbols in the name are converted to dash symbols ("-").
  • All letters are converted to lower case.
  • “HTTP_” prefix is removed

Special naming convention! Why? We are already writing in XML. Is that not special enough. Especially when you consider that adding headers to outbound response in the web.config is declared as you would expect. So what they require is this format:

    <set name="HTTP_X_PRERENDER_TOKEN" value="{my-secret-token}" />

So the complete syntax that is required to add the header to the outbound request in the IIS rewrite rule is:

<rule name="Prerender" enabled="true" stopProcessing="true">
    <serverVariables>
        <set name="HTTP_X_PRERENDER_TOKEN" value="{my-secret-token}" />
    </serverVariables>
    <match url="^(?!.*?(\.js|\.css|\.xml|\.png|\.jpg|\.jpeg|\.gif|\.pdf|\.doc|\.txt|\.ico))(.*)" ignoreCase="true" />
    <conditions logicalGrouping="MatchAny">
        <add input="{HTTP_USER_AGENT}" pattern="bingbot|googlebot|baiduspider|twitterbot|facebookexternalhit|rogerbot|linkedinbot|embedly|quora link preview|showyoubot|outbrain|pinterest|slackbot|vkShare|W3C_Validator" />
        <add input="{QUERY_STRING}" pattern="(.*)_escaped_fragment_(.*)" ignoreCase="false" />
    </conditions>
    <action type="Rewrite" url="https://static-ssl.prerender.io/https://www.my-spa-site{REQUEST_URI}" appendQueryString="false" />
</rule>

Another day shaving the yak.