“User-Agent” Headers Holding Back The Web

Every time you visit a website the name and version of your browser is sent to the service. In fact with every requested image, video, and style sheet the same data is sent again and again. This not only wastes bandwidth, it also subtly encourages web makers to rely upon it as a shortcut to make services work consistently across platforms. Later browsers then include more tokens in their “User-Agent” header to maintain compatibility with these fragile services. Over time the header becomes larger and the web more brittle. For example, Internet Explorer 11 identifies itself as “Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko”. Can you tell which part communicates that it is Microsoft’s Internet Explorer?

Of course it’s impractical for every web site/service to test every possible combination of browsers and platforms. So those of us developing sites and services only test the most popular browsers at the moment. Over time this leads to a web which caters to a mix of the most popular browser of the past and present, depending upon the time any given service was last made. As more and more devices leverage HTTP for the Internet-of-things this problem may grow more complex. Web standards and feature detection can help.

With well defined standards and run-time detection of features it’s possible to avoid the trap of ‘sniffing’ the browser from it’s UA headers. And while cutting edge features and services may benefit in the short-term from taking the shortcut of browser detection, they can also leverage vendor-specific prefixes of features in flux. Once standardized the prefixes can be replaced with official and non-prefixed names.

My experience detecting significantly different platforms such as mobile or internet-of-things (IOT) devices do still have some valid uses for the UA header. But ultimately they may be better served by a new, simpler header or more platform-independent designs. Until then Mozilla’s recommendations are a reasonable place to start.

In recent years even the once-dominant Microsoft notes the weaknesses and problems with UA headers. Sadly, my experiments sending an empty or minimal UA header have found too many sites broken to recommend the approach to non-technical users.

How about you? What do you think of UA headers?