January 31, 2019

Creating real server-side 404s with an Emberjs single page app

Written by Andy van Stokkum. 4 min read

While there are some great features of single page web applications, there a few downsides. Apart from the latest and greatest new SPA framework getting released every few weeks, the other annoying behavior is that out of the box, most implementations don’t really support an HTTP 4o4 error code.

The reason for this is that no matter what URL or page route you request from the web server (you know - the computer in the cloud that listens on 443 and can render < and > really fast!), the web server is configured to return index.html with a status code 200. This is because the SPA is started by loading index.html and the web server typically has no knowledge of the routes defined in the client application.

Why is this a problem? For starters the organic search results go down hill. Any URL requested will apparently exist, but will actually be the SPA page-not-found/soft-404 page. Bots will walk the site and will find results for pages that were previously unpublished/deleted.

A site I was recently working with uses Emberjs. Emberjs can be extended to support server-side rendering using Emberjs FastBoot which can return 404s, but this will require the build and application configuration to be updated and will also require Node.js to be installed on the web server.

Depending on the organisation you work for, running, supporting and patching yet another platform on the web server is likely to raise the hackles of the Ops team. We were already using ASP.net on the web server so it seemed sensible to make the most of that framework.

So for the server to return either a 200 or 404 when the site loads, it needs to know about all the available routes. The site I worked on had routes determined in two places; client defined routes and CMS defined routes.

The approach was:

Update the Emberjs build process to emit a file with all the client routes e.g. client-routes.json and package it into the build artifact (Thanks Jeff!). The routes include a wild card which can match include dynamic paths such as ids.

route-definitions.json
["home","search","login","login/help","login/help/*", "open-a/*/*"...]

Create a .net web API controller in that reads both the client routes (from the deployed file) and from the CMS (umbraco in this case) and has one GET method called ResolveClientRoute(string route).

[Route("api/content/resolve-route/{*requestedRoute}"), HttpGet]
public async Task<HttpResponseMessage> ResolveRoute(string requestedRoute)
{
    if (string.IsNullOrEmpty(requestedRoute))
        return GetIndexPage(HttpStatusCode.OK);

    // In case of failure, default the statusCode to 200
    var statusCode = HttpStatusCode.OK; 

    try
    {
        var routeType = await _routeResolverService.EvaluateRoute(requestedRoute);

        statusCode = routeType.In(ResolvedRouteType.ClientRoute, ResolvedRouteType.CmsRoute)
            ? HttpStatusCode.OK
            : HttpStatusCode.NotFound;

        _systemLogRepository.Info($"Resolved route {requestedRoute} as {routeType} with status {statusCode}", GetType(), Category.Content);
    }
    catch (Exception ex)
    {
        _systemLogRepository.Error($"Error resolving route {requestedRoute}", GetType(), Category.Content, ex);
    }

    return GetIndexPage(statusCode);
}

private HttpResponseMessage GetIndexPage(HttpStatusCode httpStatusCode)
{
    try
    {
        // read index.html from disk and return as a string
        var indexPageContent = _webClientFileRepository.ReadIndexHtmlFile();

        var response = new HttpResponseMessage
        {
            Content = new StringContent(indexPageContent),
            StatusCode = httpStatusCode
        };

        response.Content.Headers.ContentType = new MediaTypeHeaderValue("text/html");
        response.Headers.CacheControl = new CacheControlHeaderValue
        {
            MaxAge = _clientAppCacheTimeSpan
        };
        return response;
    }
    catch (Exception ex)
    {
        _systemLogRepository.Error("Error loading index.html", GetType(), Category.Content, ex);

        // redirect them to the root of the site
        var response = new HttpResponseMessage(HttpStatusCode.TemporaryRedirect);
        response.Headers.Location = new Uri("/", UriKind.Relative);
        return response;
    } 
}

For any route requested, it checks to see if the route exists. The HttpResponse returned will always contain the contents of index.html and the status code of 200 or 404 depending on if the route was found.

public class RouteResolverService : IRouteResolverService
{
    public async Task<ResolvedRouteType?> EvaluateRoute(string requestedRoute)
    {
        // Check the client routes (these should be cached)
        var clientRouteExpressions = GetClientRouteExpressions();

        var clientRouteExists = clientRouteExpressions.Any(regex => Regex.IsMatch(requestedRoute, regex, RegexOptions.IgnoreCase));
        if (clientRouteExists) return ResolvedRouteType.ClientRoute;

        // Check the CMS routes
        var cmsRouteExpressions = GetCmsRouteExpressions();        
        var cmsRouteExists = cmsRouteExpressions.Any(regex => Regex.IsMatch(requestedRoute, regex, RegexOptions.IgnoreCase));
        if (cmsRouteExists) return ResolvedRouteType.CmsRoute;

        return null;
    }
    
internal IReadOnlyCollection<string> GetClientRouteExpressions()
{
    if (string.IsNullOrWhiteSpace(_settingsProvider.WebClientDirectory)) throw new ArgumentException(nameof(_settingsProvider.WebClientDirectory));

    var clientRoutesJson = _webClientFileRepository.ReadClientRouteFile();
    var clientRoutes = JsonStringHelper.CreateObjectFromString<string[]>(clientRoutesJson);

    var clientRouteExpressions = clientRoutes.Select(route => CreateRouteRegex(route));
    return clientRouteExpressions.OrderBy(route => route).ToList();
}

internal string CreateRouteRegex(string route)
{
    // remove trailing 
    var routeParts = route.Split(new[] {"/"}, StringSplitOptions.RemoveEmptyEntries);
    // update wild card with regex
    // escape symbols such as $ which is sometimes used in urls e.g. deposit$20
    var expandedRouteParts = routeParts.Select(part => part.Replace("*", @"[^\/]+").Replace("$", @"\$"));
    // now build the regex
    var pattern = string.Join(@"\/", expandedRouteParts);
    // build full pattern and add optional trailing /
    var fullPattern = "^" + pattern + @"\/?$";
    return fullPattern;
}

Add an IIS rewrite rule to rewrite all not static file requests to the web API /resolve-client-route/{route}.

<rule name="default" stopProcessing="true">
  <match url="^$|^unavailable$" />
  <conditions logicalGrouping="MatchAll" trackAllCaptures="false" />
  <action type="Rewrite" url="index.html" />
</rule>
  
<rule name="static files" stopProcessing="true">
  <match url="^assets/.*|^favicon/.*|^moment/.*|^static/.*" negate="false" />
  <conditions logicalGrouping="MatchAll" trackAllCaptures="false" />
  <action type="None" />
</rule>
 
<!-- site routes e.g. any resource that is not static  -->
<rule name="site routes" stopProcessing="true">
  <match url="^([^\.]*)$" negate="false" />
  <conditions logicalGrouping="MatchAll" trackAllCaptures="false" />
  <action type="Rewrite" url="http://localhost:80/api/content/resolve-route/{R:0}" appendQueryString="false"/>
</rule>

The browser will receive the status code and the SPA will either show the requested page or will display the page-not-found page.