So, You're Writing A Responsive Images Script
April 2020 note: Hi! Just a quick note to say that this post is pretty old, and might contain outdated advice or links. We're keeping it online, but recommend that you check newer posts to see if there's a better approach.
Hey, all the cool kids are doing it.
In a responsive site, images need to scale gracefully across a huge range of viewport widths and screen densities. On the surface, this is easy enough: setting img elements to max-width: 100% in our CSS allows the images to scale down from their inherent size to suit smaller layouts.
The trouble with this approach is that it requires the image source to be at least as large as the largest size at which it will be displayed. If an image is intended to be displayed anywhere from 300px wide to 2000px wide, the image source must have an inherent width of 2000px or greater. We wouldn’t want to scale an smaller image up to 2000px wide, since it would become distorted.
While this approach works well visually, it comes at a major cost to the user: scaling an image down with CSS obviously doesn’t reduce the bandwidth cost involved in loading it. In a responsive layout designed to work everywhere from wide-screen desktop displays to mobile phones, max-width: 100% means sending the smallest displays an image that, while scaled down so it looks correct, exacts the same bandwidth cost as on a desktop display. A 300px wide layout would be much better served by an image source with an inherent width of 300px, since the user will get an identical effect without the additional bandwidth overhead.
There’s an especially clever way of working around this concern with a combination of oversized images and high compression ratios, but this comes with complications of its own—for example, increased memory consumption on the user’s device, and distorted images when saved by the user. Many times, a compressive image will be enough to get the job done without any custom scripting—but there are many use cases where scaling down an image won’t be enough.
When you hear the phrase “responsive images,” it isn’t usually in reference to the simple flexible images inherent to responsive web design. “Responsive images” has come to mean “a method of dynamically swapping image sources with versions that are most appropriate for the user’s context.” The primary goal is to avoid sending an image source large enough for the largest layouts to users viewing smaller layouts, for the sake of conserving the user’s bandwidth. Secondary goals might include serving high resolution images only to users with high resolution displays, or providing sources with alternately cropped and scaled images to better highlight the subject depending on how it’s being displayed.
The Problem
Permalink to 'The Problem'Swapping an image’s src certainly sounds simple enough. There are scores of responsive images scripts that do exactly that, and that alone: an img with an src, and an alternate src in a data- attribute to be swapped in depending on conditions hard-coded in a script. In fact, that’s how our original responsive images script worked.
But this approach is not optimal: any time we change the src on an image, we’re fighting against the browser’s natural behaviors. There are layers upon layers of optimizations built into browsers to pre-parse, pre-fetch, and pre-render images long before we even see a page begin to load—even as far back as Internet Explorer 8. Some of these processes can kick off preemptively based on the assumption that we might navigate to a page.
As detailed in full gory detail elsewhere, we tried a few tricks trying to work around some of this prefetching behavior, but didn’t have much luck in the long term—this behavior isn’t standardized, and each browser does things a little differently. Even then, we were working to undo major browser-level performance optimizations. Ideally, a solution would be able to take advantage of these features.
That’s what started us down the road to getting responsive images baked into the HTML5 specification: a standard solution that could take advantage of browser optimizations, rather than dodging them or working against them outright. That standard has been a long time coming—a huge group of developers (us included) has been working toward that solution, and we’re making steady progress, but it’s in process.
But this is an issue we need to address right now in our day-to-day work. So in the interim, we (and many others) are using Picturefill to bridge the gap, which mimicks the proposed standard using markup that complies with current standards—span, img, and data- attributes.
Still, it seems like a new “responsive images” script comes along every few days. The specific features that a project needs can vary, which can make a case for building something from scratch. For that matter, the topic still feels like it’s “up in the air,” especially when the search for a native solution involves so much public brainstorming and feedback.
Potential Pitfalls
Permalink to 'Potential Pitfalls'Now, I don’t intend to discourage anyone from writing a new responsive images script, by any stretch of the imagination. Each one that comes along helps us all understand the problem a bit better, and arms standards efforts with more data on what features and syntaxes are most important to developers.
In fact, to some extent, the same document we’re using to keep tabs on the issues that impact developers and users the most can be used as a rough checklist for any open source responsive images solution. There’s a lot of information to parse through in that document, though, so let’s break it down to a list of the biggest concerns when we’re implementing a responsive images solution using existing technology:
Does it make duplicate requests?
Permalink to 'Does it make duplicate requests?'This is a common issue, and one we’ve already discussed above—if an img starts out with a real src, there isn’t anything we can reliably do to prevent duplicate requests from happening. So, in any situation where you’ll end up needing to swap the contents of an src attribute that already points to a source—say, changing out a “mobile”-sized image source for a larger version—you’ll end up making two requests for each image displayed. Suddenly we find ourselves doing more harm than good for some users.
Further, there’s a surprisingly common bug in older browsers where an img with an empty src will trigger a request to the root of the page’s directory—for the nonsensical URL “src” or for the page itself—resulting in an additional request and a potential data transfer for no real reason. This would be easy enough to shrug off were it not affecting browsers as common as Android 2.3, which makes up nearly 25% of Androids in use today.
So, how do we go about using img at all? Well, we ran some tests and found that while an empty src (<img src="">) or omitted src value (<img src>) can still trigger these requests, a completely omitted src attribute (<img>) would not .
So, to avoid redundant requests when using img, you may need to make use of a pattern like this in your markup:
<img data-src="" data-src-med="" data-src-lg="">
But addressing that concern brings us to an even bigger one:
What happens when JavaScript breaks?
Permalink to 'What happens when JavaScript breaks?'The number of people who have JavaScript wholesale “disabled” at the browser level is—while non-zero—incredibly small.
A broken JavaScript experience, however, is far more common. For example, a third-party script blocked by a browser plugin or containing a syntax error will bring the whole page’s scripts crashing down. If we’re using an img without a src, we find ourselves in a situation where our content is entirely dependent on JavaScript—and on “nothing going wrong.” If you’ve worked on the web for any length of time, you know this isn’t a particularly comforting scenario.
We’re kind of an an impasse already—we can’t use an img with a real src without wasting bandwidth somewhere, and we can’t keep all of our image logic in JavaScript without running the risk of a user never seeing it.
So what’s the optimal approach? The best answer we’ve been able to come up with is: “It depends.”
On a very JavaScript-heavy site with multiple third-party scripts, advertisements, and so on—also known as “multiple potential points of failure”—it may be worth incurring the extra overhead of a wasted request. In this case, you’ll want to take tremendous care to ensure that your images are as compressed as possible, to mitigate the damage to whatever context will receive the wasted request.
On a site that uses very little JavaScript, it may be best to use JavaScript to alter an src-less img, or generate the img completely from scratch. In the latter case, you’ll want to provide a fallback img with a real src in a noscript tag; the contents of noscript aren’t subject to prefetching, and it provides a fallback for that non-zero number of users with JavaScript disabled with minimal effort.
Does it support art direction?
Permalink to 'Does it support art direction?'The phrase “art direction” may not be familiar to all, especially when relate to web development. In a responsive image context, this term has come to mean “sources that make use of different cropping, zooming, and focal points depending on the current display size or breakpoint.”
We’ve found this to be a very common request in our client work, and one that’s at-odds with the idea of automating alternate sources from a single image: e.g., if alternate size image sources are generated from a single uploaded image, we lose the ability to tailor our image for different display contexts—and a wide-angle shot of a subject like the desktop image above becomes an incomprehensible blur at smallest sizes.
We’ve seen a few solutions that support a sort of “overflow” viewport for images, providing CSS-based cropping and zooming across breakpoints. This approach will, however, require a larger image than the user needs, and result in some bandwidth waste. And depending on the project or client process, we might find this approach somewhat limiting in terms of real-world application: imagine a disclaimer above the upload field in your CMS instructing users to only upload images that have a specific focus in one of X areas, or needing to add a series of controls for highlighting the area of the image that should have focus across a given set of breakpoints depending on the image’s position in the layout? Suddenly, all our automation has made things a lot more complicated than simply uploading multiple versions of an image.
Will it work in the real world?
Permalink to 'Will it work in the real world?'I can personally attest to putting together a number of responsive images tests cases, some of which were pretty clever (if I may say so myself). The cleverer they were, the more apt I was to start showing off isolated demos and test cases right away. By my estimation, I’ve “solved” responsive images a handful of times now—at least, within the scope of a carefully assembled demo where I control every asset involved.
Unfortunately, there seemed to be a direct correlation between the “cleverness” of these solutions and them ending up as novelties, rather than something we could put into use on real client work. The breakdown comes when a project leaves your hands and passes off to a client for integration with their CMS of choice.
- If your solution depends on breakpoints hard-coded in your JavaScript, it can’t be easily overridden when new pages and sections are added to the site. You (or, more accurately, your clients) end up with a stylesheet, of sorts, hard-coded in a JS file.
- If your solution depends on CSS to manage image sources—especially where it feels like the natural place for breakpoint-dependent logic—that stylesheet will need to be updated by the CMS every time a new image is uploaded to the site.
Unfortunately, anything beyond a hand-coded artisanal HTML file is going to require a high degree of automation somewhere down the line. Any solution—even an especially clever one—that doesn’t line up with an automation-friendly process ends up being a trinket.
If you haven’t already noticed from the tone of this section, coming up with too many of these can be a quick path to bitterness.
When it fails, are you aware of where will it do the most damage?
Permalink to 'When it fails, are you aware of where will it do the most damage?'Every responsive images solution will fail somewhere, and not just in the ways we’ve already discussed—those are the ways we know about. If our solution depends on anything but the most tried-and-true features to function, we have to accept that it’s going to break in some browsing environments due to lack of support for something we’ve decided to leverage.
For example, lots of proposed solutions involve using bleeding-edge standards to create a responsive images “controller” of sorts. Doing so means we may be opting scores of users out of our site’s content based on the age of their browsers.
As a defensive measure, we always advocate a “mobile-first” approach to responsive images—starting with the image most appropriate for a small screen, and conditionally “enhancing” to a larger version for large screens. Mobile browsers have wildly varying support for features, and are the context where wasted bandwidth stands to do the most damage. We grant that we can’t ever make assumptions about a user’s current bandwidth based on the size of a user’s screen, but we can pretty safely say that a mobile device is more likely to be a browsing environment where bandwidth is at a premium. In the event that anything should fail, even users on desktop browsers will be left with a representative image, if not one ideally suited to their layout.
When a native solution is settled on once and for all, will it get out of the way?
Permalink to 'When a native solution is settled on once and for all, will it get out of the way?'The benefit to a native solution goes well beyond ease of use. Browser-level performance optimizations might include outfitting mobile browsers with a “request low-resolution images” option for limited bandwidth contexts, or methods to avoid re-requesting images when resizing a browser window down (as replacing larger image sources with the identical smaller versions will mean additional requests with no visible benefit).
Any solution we come up with for responsive images is likely to be a stop-gap. While this doesn’t mean we should sit on our hands and wait for someone to solve this for us—especially where we’d be doing so at our users’ expense—we should plan accordingly. In a few years, even the most clever “right now” solution to responsive images is going to be an inscrutable hack to anyone viewing source. We have to plan for our solutions’ obsolescence.
We use Picturefill in our day-to-day work, but never the version that makes use of the actual picture element—we simply mimic the proposed pattern with current standards. While this might look strange to someone years from now, having only known the standard for responsive images, it’s a far better situation than using the picture element itself and running the risk of our prollyfill’s behavior not matching up with the native element’s behavior. Once picture has been added to a few of the major browsers and the spec stabilizes, Picturefill will be adapted into a fully featured polyfill.
Discouraging?
Permalink to 'Discouraging?'It sure sounds that way, at face value—but stick with it. Look at it this way: all the concerns above are equivalent to a well-populated issue tracker in an open source repository. Each person that has worked on this topic has helped to prove, disprove, refine, and define all the gotchas and caveats we’ve mentioned here—and we’re still only scratching the surface. It’s a tricky topic to leap into head-first, and you’re apt to run into many of the same issues and false starts as the developers before you, but in doing—and publishing the lessons we learn along the way—we better define the issues that need to be solved.