Measuring Multi-CDN Performance with Resource Timing Data

If you’re using RUM to tell you how fast your page is loading, you’re only getting a portion of the story; just a headline. Modern browsers widely support Navigation Timing API and as well as Resource Timing API. The possibilities with this data are endless. With distributed teams focused solely on their respective products, resources like CDN are seen as ancillary to the overall page load, however, improperly referenced objects can wreak havoc on your end users.

This post will focus on expanding your RUM solution to isolate and measure CDN delivered content and provide a strategy for comparing individual CDN performance on provider-agnostic hostnames.


CDN Steering and Agnosticism

If you’re only using one CDN provider, then this post may not do much for you. However, if your site is running on multiple CDNs around the world, you may be interested in digging deeper into which CDNs are providing the most benefit to your users. Additionally, you may want to be more informed about your steering decisions.

First, let’s evaluate a few steering options:

Finite Hostnames – you benefit from the ability to granularly steer your users on a user-to-cdn performance basis. This is good if browser caching is not a concern to you, i.e., you have highly dynamic content.

DNS Round-Robin (aka, CDN Roulette) is crap. Don’t do it. If you have to, then you will want to know if your weighted records are having the desired effect. Also important, if you are using DNS Geo Records, are your users being steered to the right CDN or is your DNS provider making mistakes?

Dynamic Steering using RUM and CDN-agnostic hostnames is what all the cool kids are doing these days. Dynamic Steering platforms do their jobs well, but if you’re looking to show their benefit via your RUM instrumentation, you will need to be able to provide granular metrics on how CDNs are performing.

For any of the above methods, the ability to identify and report on CDN performance is necessary for making informed business decisions.

A Note on Resource Timing

Resource Timing provides a ton of additional insight into the objects that were served during page load. Rather than diving too deeply, simply take a look at the Resource Timing Level 1 Model below and start thinking about the things that are important to you, your users, and how this data can be used for your benefit.

https://www.w3.org/TR/resource-timing-1/#processing-model

Of the above metrics, the ones that can be derived are more interesting and useful for measuring CDNs than the ones presented.

  • Useful Metrics
    • Browser Caching: Whether or not the object was fetched from the CDN.
      (resource.transferSize == 0) ? true : false
    • Time to First Byte: On content fetch, what was time to get the first byte of the server response.
      resource.responseStart - resource.startTime
    • Secure Connection Time: Provides TLS handshake timing information.
      (resource.secureConnectionStart > 0) ? (resource.connectEnd - resource.secureConnectionStart) : 0
    • DNS Lookup Time: Time for the browser to lookup a cold DNS record.
      resource.domainLookupEnd - resource.domainLookupStart

The Power of X-Headers

Accessing Resource Timing information is great, but how it’s correlated to the respective CDN is what we’re really shooting for. After all, there’s no benefit to pulling granular data if you can’t turn it into useful information.

Headers already serve a very useful purpose by providing instructions for the browser and its respective RFC implementations on how to handle content despite a lack of uniformity across browsers. In addition to the RFC defined headers, there exists a type of header can be customized: the X-Header.

Despite some misconceptions, X-Headers are not necessarily deprecated; they are merely no longer acceptable for new protocols. X-Headers have historically been used to distinguish permanence, however, some have crept their way into widely used protocols as standard headers, such as “X-Sender” in email. But X-Headers can be used for a variety of other things. For example, a CDN may utilize an X-Cache header to display a resource’s cache status for debugging and troubleshooting.

Almost all CDNs and proxies provide the ability to customize inbound and/or outbound response headers. With this, identifying which CDN served a request becomes fairly trivial. The result is a rule in your configuration that looks like this Varnish rule:

sub vcl_fetch { 
  set resp.http.X-CDN = "CDN-A";
}

Exposing X-Headers with CORS

X-Headers alone are incredibly useful for troubleshooting, but now we’re stuck with a resource with additional headers and a browser that limits what we can access. CORS to the rescue.

Cross-Origin Resource Sharing, or CORS, is a specification designed to enable resources and their data to be shared across “origins” or hostnames. An example of this is an XmlHttpRequest executed by javascript on traffiq.com attempting to access a resource from cdn.traffiq.com. Since they are not the same “origin” and depending on the resource type, most modern browsers will restrict what information is accessible, if at all. With this in mind, we now see that simply using AJAX to access X-headers requires just a bit more work.

To access more information, you simply need to add a few more headers. First, we’ll add Access-Control-Allow-Origin, which basically tells the calling User-Agent which origins may access that resource. Second, we’ll add Access-Control-Expose-Headers with a list of headers that we’re giving the User-Agent access to. Finally, for the sake of completeness, we’ll also add Timing-Allow-Origin. Timing-Allow-Origin is actually part of the window.performance.timing API, but is functionally similar to Access-Control-Allow-Origin.

The same process for adding the X-CDN header above can be followed here. Once added, your response headers should resemble the following:

HTTP/1.1 200 OK
Date: Wed, 26 Apr 2017 21:47:45 GMT
Last-Modified: Fri, 10 Mar 2017 06:06:49 GMT
Content-Length: 42
Content-Type: image/gif
Access-Control-Allow-Origin: *
Timing-Allow-Origin: *
Access-Control-Expose-Headers: X-CDN
X-CDN: CDN-A

Bringing It All Together

Now we have some headers that identify a CDN and we’ve lifted restrictions on what can be done with them. But via Resource Timing API, we won’t be able to access more than performance data. Meaning those CDN identifying headers aren’t really going to do much good. Or will they?

Let’s think about what we want: Resource Timing correlated to CDN.

Here’s a breakdown of how we’re going to accomplish this. First, we need to pull the resource timing for all of our objects. Then, we need to determine which objects are CDN objects; we can either use a regular expression or a finite list of hostnames. Finally, we’ll want to use an uncached CDN resource to determine which CDN served our page load. The javascript below is an example of how this can be achieved.

Sample Code for Javascript Beacon

// helper function to extract hostname
function extractHostname(url) {
 var hostname;
 if (url.indexOf("://") > -1) {
   hostname = url.split('/')[2];
 } else {
   hostname = url.split('/')[0];
 }
 hostname = hostname.split(':')[0];

 return hostname;
}

// helper function to define which hostnames are CDN hostnames
function isCDN(hostname) {
  return hostname == "cdn.traffiq.io";
}

// get the resource timing objects and initialize our curated list
var resources = performance.getEntriesByType("resource");
var cdnResources = [];
for (var i = 0; i < resources.length; i++) {
  if (isCDN(extractHostname(resources[i].name))) {
    cdn_resources.push(resources[i]);
  }
}

var xhr = new XMLHttpRequest();
// pick a CDN object and extract headers.
var cdnObject = cdnResources[0].name;
// alternatively, use a beacon
// var cdnObject = "//cdn.traffiq.io/beacon.txt?unique=" + Date.now()/1000;
xhr.open('GET', cdnObject, true);
xhr.send();
xhr.getResponseHeader("X-CDN");

A beacon object can be used in lieu of an object that has already been served. Resource Timing can tell us if an object is browser cached or not, but a beacon with a unique query string is definitive. The caveat here is that a beacon MUST BE a) small, b) cached at the CDN edge and c) unique enough to not be cached by the browser. Unique objects require additional setup, but the payoff is accuracy.

Conclusion

Integrating CDN identification into your current RUM implementation should be fairly straight forward and the tips above will provide some clarity for better tracking and troubleshooting your multi-CDN strategy.