An Ajax-Include Pattern for Modular Content
April 2020 note: Hi! Just a quick note to say that this post is pretty old, and might contain outdated advice or links. We're keeping it online, but recommend that you check newer posts to see if there's a better approach.
While developing the front-end of the BostonGlobe.com site last fall, one of the toughest challenges we faced was delivering roughly the same content to all devices (and connection speeds) while ensuring the most important content on a page was usable as soon as possible. We approached this challenge with a variety of techniques, such as only loading the most essential JavaScript up-front (weighing roughly 4-6kb tops) and lazy loading the rest, dynamically injecting advertisements, and loading “nice-to-have” content via JavaScript—all after the initial content was delivered and usable.
Bullet-proofing our Content Includes
Permalink to 'Bullet-proofing our Content Includes'The idea of conditionally or lazy loading content is not a new one; indeed, Jeremy Keith covered a similar pattern last year, and many sites already employ similar techniques. These techniques help a great deal when optimizing a page, but—particularly when dealing with conditionally loading HTML content—we need to be careful to use them in ways that do not introduce barriers to universal accessibility.
In order to lazy load nonessential content on BostonGlobe.com without introducing access barriers, we developed a simple, markup-driven “Ajax-include” pattern that enhanced certain links on a page by including a fragment of content from a URL related to their linked resource. For example, the header of BostonGlobe.com contained a link to the Weather section of the website, and that same link is also used as a marker for including a fragment of content from that section that is used to create a weather widget.
Initial State:
Permalink to 'Initial State:'
Expanded, Ajax-enhanced state:
Permalink to 'Expanded, Ajax-enhanced state:'
With this pattern in place, we found a nice balance between loading performance and accessibility: the content begins lightweight, and accessible without Javascript, and it is enhanced into a richer experience on capable devices—all without holding up the initial page load.
From a technical standpoint, the approach was simple: any element in the
page could reference an external HTML fragment by URL through a small
set of predefined HTML5 data- attributes; our JavaScript would simply
loop through those elements, Ajax-request their referenced content, and
inject it into the page (either before, after, or in place of the
referencing element, depending on which data- attribute was in play).
Pseudo-markup for the particular scenario above, which the JS would enhance by including the referenced fragment:
<a href=”fullpage.html” data-include=”fullpage-fragment.html”>Full Page</a>
The technique worked well, and we began using it in several places to lighten the Globe’s initial load. Around that time, we also tweeted a gist snippet of the technique as it stood (crediting Scott Gonzales for some of the initial thinking behind the idea).
A tipping point
Permalink to 'A tipping point'Unfortunately, as we began to use this technique more within our templates, its effectiveness began to decrease: each Ajax-include incurred a separate HTTP request, and with more than a few includes on a page at a time, those requests often clogged the time it took for a basic experience to become enhanced. On many mobile devices, this problem could quickly become serious: you only need to read a couple Steve Souders posts to know that the number of concurrent requests allowed by devices can be quite low. With varying connection speeds, it often resulted in our “nice-to-have” content taking too long to arrive.
This limitation meant restricting our usage to a few particularly worthwhile components per page, and continuing to strive to keep the total page weight as low as possible. (Which is, of course, a good plan in any case). Still, we wondered if the concept could be extended somehow without the drawbacks.
Getting-by with a little help
Permalink to 'Getting-by with a little help'Skip forward a few months to yesterday: Nicolas Gallagher and I were chatting on Twitter about dynamic content inclusion, particularly the non-essential parts:
@necolas agreed. In those situations, I like the idea of something like an anchor-include pattern for nice-to-haves. gist.github.com/983328
— Scott Jehl (@scottjehl) March 27, 2012
@necolas Thanks! working on a pattern to bring those under 1 lazy request right now…
— Scott Jehl (@scottjehl) March 27, 2012
After a bit of hacking, it appeared that our existing Ajax-include approach could be extended to handle all content includes via a single HTTP request. Not long after that, a Gitub repo was pushed, which offers the code for you to use anywhere you’d like, and perhaps help us improve too (if you please).
A Quick Demo
Permalink to 'A Quick Demo'The page linked below contains with 3 links to hypothetical,
non-essential content. Data-after attributes in the page reference
content by URL as well, which is pulled in dynamically and automatically
with Ajax. If you use a browser inspector tool, you’ll notice that the 3
Ajax-include’d sections of the page arrive via one request.
Demo page: https://filamentgroup.com/examples/ajax-include/demo.html
Github code repo https://github.com/filamentgroup/Ajax-Include-Pattern/
How it works
Permalink to 'How it works'The difference between the initial approach and this single-request version is not major: instead of requesting each piece of content via separate HTTP requests, the JavaScript makes a single request to a server-side helper script (included in the code repo), telling it which pieces of content to pack into a single response back to the UI.
Technically, all this means is a PHP-or-otherwise script is configured
to accept a query string of comma-separated URLs and spit out the
contents of those files in a single response. To identify the separate
pieces of content within the response, the PHP proxy wraps each one in
an invented <entry> element with a url attribute referencing the
location from which it was fetched (note: this was formerly a
<page> element, but now that our demo is using
quickconcat, it’s using
<entry>. As such, the JS is now updated to look for <entry> element
elements.). When this response arrives back to the UI, the JavaScript
notifies all of the Ajax-include elements on the page, and the proper
subset of content for each element is injected into that element’s
location in the page.
The proxy, by the way, is optional; without it, the script will revert to multiple requests. Depending on the use case, this might be preferable as well. For more information on the proxy that we’re using in the demo (and that is included in the code repo) check out the quickconcat project on Github.
As far as dependencies go, the current script uses jQuery, but it could probably be done pretty concisely without it (though the event handling might make for some fun workarounds). It could also easily be built without PHP. We would love to see a pull request or two with alternatives!
Using the code
Permalink to 'Using the code'To use the technique yourself, grab the code from the Github repo above and…
-
Include jquery.js and jquery.ajaxinclude.js in your page (anywhere’s fine)
-
Add attributes to elements in your page where nonessential fragments of content can be included from an external URL, using jQuery-api-like qualifiers like
data-after,data-before, anddata-replace:<h2 data-after="/related-articles/45?fragment"><a href="/related-articles/45">Related Articles</a></h2> -
When the DOM is ready, select the Ajax-include elements on the page and call the ajaxInclude method on them, passing the URL of the server-side proxy and expected query string:
$( "[data-after]" ).ajaxInclude( "quickconcat.php?wrap&files=" );
Note: an ideal use case might have an anchor link somewhere in the page referencing a full HTML document, and a data-attribute somewhere else in the page referencing a smaller HTML fragment to include into the page (pseudo-coded above with the ?fragment in the URL). You might also employ smarter server logic to simply return the fragment when the page is requested via Ajax instead of regular HTTP, but let’s not get ahead of ourselves…
For more information on using the code, check out the project readme.
Taking this pattern further
Permalink to 'Taking this pattern further'The idea of a building websites in a more modular, container-independent manner is a hot topic, and for good reason: every day we’re dealing with more diversity across the devices that access our sites, and we want to deliver the best experience we can to all.
With the downsides of this pattern tamed, perhaps we could even start to consider ways to extend our notion of what’s essential in a webpage…
Perhaps a news article’s initial HTML delivery consists of nothing but the article itself and some links to the top-level sections of the website, while the rest of the page is deferred to Ajax-includes… or maybe a landing page consists of nothing but links to the top-level sections of a website, and then the rest of the content (headlines, article teases, etc) is included via ajax in a separate request.
In both of these examples, the page would deliver light-weight and lightning fast, allowing you to prioritize the truly essential bits that should be there at the start, and let the slow stuff come in moments after.
You might even take it farther to apply a DRY-like pattern across all pages on a site or application, where each piece of content truly has only one home, and pages can borrow freely from the content of others to create mashup pages that resemble the sorts of pages we use today. (And perhaps a slightly smarter server-side helper could find the fragments within a page by their identifiers to facilitate this…)
A related sketch from my notes during last year’s Mobilewood meetup (click image for full size):
Thoughts? Ideas?
Permalink to 'Thoughts? Ideas?'Whether you do or don’t use this particular script, we’d love to hear what you think of the pattern. We’re pretty excited by the potential optimizations that could come out of it, so please drop us a note, or even fork the code and extend it further!