How much data should my Service Worker put upfront in the offline cache?

I love when Web site/apps work even when I’m offline. I’ve made my SVG game esviji work offline thanks to appcache just after attending Jake Archibald conference about why Application Cache is a Douchebag during the 2012 edition of the Paris Web conference. Fortunately, we have now Service Workers (in some browsers), which gives us more control over this kind of cache for offline browsing. But as Uncle Ben says, “With Great Power Comes Great Responsibility”.

Just like with appcache, it is possible with Service Workers to put a full website in the cache when loading the first visited page.

It is very interesting, because you can then go offline and browse the whole site just as if you were online, without even noticing you’re offline. The cache will then be updated when you visit pages of the site while online. Depending on the nature of the content, you will fetch the page from the server when it is requested by the user, so that she gets the up-to-date version1, or you will show the cached version first, and update the cache only for subsequent visits.

All of this is really well explained by Jeremy Keith in a series of posts on his blog, including the recent one about Making Resilient Web Design work offline.

Resilient Web Design is a Web book Jeremy wrote a few weeks ago. I urge you to read this book, it’s really great. Just like most of Jeremy’s creations, anyway.

Here’s an extract of the book’s introduction:

The World Wide Web has been around for long enough now that we can begin to evaluate the twists and turns of its evolution. I wrote this book to highlight some of the approaches to web design that have proven to be resilient. I didn’t do this purely out of historical interest (although I am fascinated by the already rich history of our young industry). In learning from the past, I believe we can better prepare for the future.

So, back to the topic of this post.

Jeremy had the great idea to make this book available offline thanks to a Service Worker, so you can visit it once, even only one page of it, and read the whole book while offline, commuting like me in Paris underground subway for example2.

This is great! There is a lot to come for the Web thanks to such features, assembled in Progressive Web Apps3.

But, it means Jeremy chose to fetch the whole site content and resources in every capable browser4, even if the user only wants to read the introduction, and decide that she doesn’t need to read the rest. I would call her a fool, of course, but it might happen.

According to my browser network panel or WebPagetest, it means almost 16 Mb are downloaded right away when you access one page of the site.

The Resilient Web Design web book audited by WebPagetest

The site is very fast, and all checks are green, but that’s because most of the downloads happen asynchronously, after the visited page has been rendered.

I must confess I did almost the same thing for a while in my game esviji when I started using appcache, because I put almost 2 Mb of audio files in the cache. I decided later that offline users could play without sound, so I removed it from the cache.

For a small site/app that takes 2 or 3 Mb, I can accept to download everything, because the convenience to have all this available while offline can be great. But I think 16 Mb is really to much.

Just to illustrate, it means that one visit to this site will cost a Mauritanian at least 10 % of his daily income, according to Tim Kadlec’s simulation on What Does My Site Cost?.

Cost of visiting this website as a percentage of daily income

Only 0.24 % for Jeremy in UK or 0.28 % for me in France, but we are here because we love the World Wide Web, not Wealthy Westerners’ Web, as presented by Bruce Lawson during 2016 edition of the Paris Web conference.

Because I use it quite a lot these days to check my own Progressive Web Apps, I thought it would be nice if Lighthouse, the Chrome extension that check web pages against a growing list of best practices, included a check on total page weight. It looks like Hubert Sablonnière already had this idea and created an issue, which got support from Paul Irish, so it will come sooner or later.

For my own website, I first thought I would only cache visited pages. But I now cache the homepage, the two about pages, and the last post, regardless of the page on which the user arrives, for a really light total weight of 87 KB additional resources. The offline fallback page lists the pages that are in the cache, so that the user can discover some unknown content even when she’s offline. This is a WIP, so it might break, and it will change over the coming weeks, because I might adjust my strategy.

There is a user setting to “save data” in some browser, which activation adds a new HTTP header we can test in our Service Workers, as shown by Dean Hume in his post Service Workers: Save your User’s Data using the Save-Data Header, but I think most people that are not as tech savvy as us will never notice this setting, so it’s obviously a nice to have, but it’s not enough.

So, it might be nicer to initially cache only the files needed to enhance the performance of the site and provide a clean offline fallback, then add the pages when they are visited, and provide the user with an option to cache the whole site, or part of it, for future offline browsing.

It would be less magical, indeed, but more respectful of users with limited and/or costly data plans.

I don’t know if Jeremy thought about this or not, but I hope there will be some discussions around this in the community, because Service Workers give us a lot of power, that could be abused by people not aware of the damages it can cause, or even on purpose, just because it helps making websites faster. When the average page is already more than 2 Mb, we really have to be careful.

To conclude, it’s kind of amusing to see that Jeremy also provides links to download other versions of the book, including PDF, epub and mobi, and most of these files weight less than 16 Mb.

February 25, 2017 update: Lighthouse will now give a lower score if total byte weight is too high.

  1. Be careful, you can still get a not so up-to-date version if the page is taken from the traditional browser cache. Yes, “it’s complicated” sometimes, as shown in this awesome post written by Yoav Weiss⬆︎

  2. Well, that’s pure fiction, because I have an iPhone, and Apple didn’t work yet on supporting Service Workers in iOS. Just like Scott Jehl, “As an iOS user, the lack of Service Worker support in Safari is almost enough for me to switch to Android. Almost.”. ⬆︎

  3. You can read more about Progressive Web Apps in french on my company’s blog: Les Progressive Web Apps pour booster l’UX de vos services⬆︎

  4. As of today, these include only Firefox, Chrome and Opera. ⬆︎

Si vous voulez signaler une erreur ou proposer une modification de ce texte, n’hésitez pas à l’éditer directement à la source sur Github.

8 commentaires

  • Idée de bonne pratique: l’utilisateur est informé du poids qui sera téléchargé-stocké sur son cache:-) (à l’instar du poids des fichiers-joints, PDF etc.), et peu le refuser pour consultation sans stockage.

  • Ça rejoint effectivement la suggestion de Addy Osmani: ' } else { list[i].innerHTML += ' ' } } })(this) // Lazyload Webmentions avatars ;[].forEach.call(document.querySelectorAll('img[data-src]'), function(img) { img.setAttribute('srcset', img.getAttribute('data-srcset')) img.setAttribute('src', img.getAttribute('data-src')) img.onload = function() { img.removeAttribute('data-srcset') img.removeAttribute('data-src') } }) /***************************************************************** * PWA * ****************************************************************/ // Install Service Worker if ('serviceWorker' in navigator) { // https://slides.com/webmax/serviceworker-thebest/#/23 window.addEventListener('load', () => { navigator.serviceWorker.register('/sw.js') }) } // https://stackoverflow.com/a/18650828/717195 function formatBytes(a, b) { if (0 == a) return '0 Bytes' var c = 1024, d = b || 2, e = ['Bytes', 'KB', 'MB', 'GB', 'TB', 'PB', 'EB', 'ZB', 'YB'], f = Math.floor(Math.log(a) / Math.log(c)) return parseFloat((a / Math.pow(c, f)).toFixed(d)) + ' ' + e[f] } // Log storage quota usage // https://slides.com/webmax/serviceworker-thebest/#/35 if ('storage' in navigator && 'estimate' in navigator.storage) { navigator.storage.estimate().then(({ usage, quota }) => { console.log( 'Using ' + formatBytes(usage) + ' out of ' + formatBytes(quota) + ' bytes.', ) }) } // Clean Service Worker cache // https://adactio.com/journal/9888 window.addEventListener('load', function() { if (navigator.serviceWorker.controller) { navigator.serviceWorker.controller.postMessage({ command: 'trimCaches' }) } }) /***************************************************************** * Deal with offline/online events * ****************************************************************/ // https://mxb.at/blog/youre-offline/ // https://www.youtube.com/watch?v=7fnpsF9tMXc let isOffline = false window.addEventListener('load', checkConnectivity) // when the page has finished loading, // listen for future changes in connection function checkConnectivity() { updateConnectivityStatus() window.addEventListener('online', updateConnectivityStatus) window.addEventListener('offline', updateConnectivityStatus) } // check if we're online, set a class on if offline function updateConnectivityStatus() { let offlineNotificationToShow = false let offlineNotificationIcon = '' let offlineNotificationType = '' let offlineNotificationMessage = '' let offlineNotificationElement = window.document.getElementById( 'offline-notification', ) if (typeof navigator.onLine !== 'undefined') { if (!navigator.onLine) { // add 'offline' class to the body, for any CSS adjustment document.body.classList.add('offline') offlineNotificationToShow = true offlineNotificationIcon = 'offline' if ('serviceWorker' in navigator) { // If the browser supports Service Workers and the Cache API, // getting offline should be less stressful. Use a "warning" // message instead of an "error and provide a link to content // available in cache. offlineNotificationType = 'warning' offlineNotificationMessage = 'Désolé, vous ne semblez plus être connecté. Vous pouvez continuer à lire cette page, ou voir ce qui est dans votre cache.' } else { offlineNotificationType = 'error' offlineNotificationMessage = 'Désolé, vous ne semblez plus être connecté. Vous pouvez continuer à lire cette page en attendant le retour de la connexion.' } } else { // remove 'offline' class from the body document.body.classList.remove('offline') offlineNotificationIcon = 'online' if (offlineNotificationElement) { offlineNotificationToShow = true offlineNotificationType = 'success' offlineNotificationMessage = 'Vous être de nouveau connecté ! Vous pouvez reprendre une navigation normale sur le site.' } } if ( offlineNotificationToShow && !window.document.getElementById('offline-notification-static') ) { // https://stackoverflow.com/a/25214113/717195 let newOfflineNotificationElement = document.createRange() .createContextualFragment(`

    ${offlineNotificationMessage}

    `) if (offlineNotificationElement) { offlineNotificationElement.parentNode.replaceChild( offlineNotificationElement, newOfflineNotificationElement, ) } else { let mainElement = document.querySelector('main') mainElement.parentNode.insertBefore( newOfflineNotificationElement, mainElement, ) } } } } /***************************************************************** * Search * ****************************************************************/ // Utility function to get the search query from the URL query string // http://stackoverflow.com/a/901144/717195 function getParameterByName(name) { name = name.replace(/[\[]/, '\\[').replace(/[\]]/, '\\]') var regex = new RegExp('[\\?&]' + name + '=([^&#]*)'), results = regex.exec(location.search) return results === null ? '' : decodeURIComponent(results[1].replace(/\+/g, ' ')) } var algoliaLinked = false var algoliaLoaded = false var algoliaClient var algoliaIndex function onAlgoliaAvailable(callback) { if (typeof algoliasearch === 'function') { algoliaLoaded = true algoliaClient = algoliasearch(algoliaApplicationId, algoliaApiKey) algoliaIndex = algoliaClient.initIndex(algoliaIndexName) callback() } else { if (!algoliaLinked) { var algoliaScript = window.document.createElement('script') algoliaScript.setAttribute( 'src', '/assets/javascript/vendors/algoliasearchLite-3.24.6.min.js', ) window.document.getElementsByTagName('head')[0].appendChild(algoliaScript) algoliaLinked = true } setTimeout(function() { onAlgoliaAvailable(callback) }, 50) } } var $intro = window.document.getElementById('intro') var $input = window.document.getElementById('search_input') var $results = window.document.getElementById('search_results') var $currentUrl = window.location.toString() var $currentContent = window.document.querySelector('main') var $searchContent = window.document.querySelector('.search') var searchSettings = { hitsPerPage: 50, facets: '*', attributesToHighlight: 'title,tags', attributesToSnippet: 'content:20', } // A search query may come from the URL query string var queryString = getParameterByName('q') if (queryString.length > 0) { $input.value = queryString onAlgoliaAvailable(function() { algoliaIndex.search(queryString, searchSettings, searchCallback) }) } // A search query may come from the user typing in the search field $input.addEventListener('keyup', function() { if ($input.value.length > 0) { history.pushState(null, null, '/recherche.html?q=' + $input.value) if ($intro) $intro.style.display = 'none' $currentContent.style.display = 'none' $searchContent.style.display = 'block' onAlgoliaAvailable(function() { algoliaIndex.search($input.value, searchSettings, searchCallback) }) } else { history.pushState(null, null, $currentUrl) if ($intro) $intro.style.display = 'block' $currentContent.style.display = 'block' $searchContent.style.display = 'none' $results.innerHTML = '' } }) // Search callback function that shows the results function searchCallback(err, content) { if (content.query !== $input.value) { // If we receive a result for an old query, abort return } var months = [ 'janvier', 'février', 'mars', 'avril', 'mai', 'juin', 'juillet', 'août', 'septembre', 'octobre', 'novembre', 'décembre', ] $results.innerHTML = '' if (err) { console.error(err) return } var resultsNumber = content.hits.length if (resultsNumber === 0) { $results.innerHTML = '

    Aucun résultat, veuillez modifier votre recherche.

    ' return } var result, results, hit, hit_type, hit_title, hit_excerpt, hit_date, hit_tags results = '

    ' + resultsNumber + ' résultat' + (resultsNumber > 1 ? 's' : '') + ' :' for (var i = 0; i < resultsNumber; i++) { hit = content.hits[i] result = '' hit_type = '' switch (hit.type) { case 'post': case 'page': hit_type = hit.type case 'document': if (hit.collection === 'notes') { hit_type = 'note' } } hit_date = '' if (hit.date) { js_hit_date = new Date(hit.date * 1000) date_options = { year: 'numeric', month: 'long', day: 'numeric' } if (hit.lang === 'en') { hit_date = js_hit_date.toLocaleDateString('en-US', date_options) } else { hit_date = js_hit_date.toLocaleDateString('fr-FR', date_options) } } if (hit_type === 'note') { if (hit.lang === 'en') { hit_title = 'Note from ' + hit_date } else { hit_title = 'Note du ' + hit_date } } else { hit_title = hit._highlightResult.title.value } hit_excerpt = hit._highlightResult.html ? hit._highlightResult.html.value : hit._snippetResult.content ? hit._snippetResult.content.value : hit.excerpt_html hit_tags = '' if (hit._highlightResult.tags) { // Build the tags list hit_tags = '' hit_tags_number = hit._highlightResult.tags.length for (var j = 0; j < hit_tags_number; j++) { hit_tags = hit_tags + ', ' + hit._highlightResult.tags[j].value } hit_tags = hit_tags.replace(/^, /, '') } result = '

    ' + hit_title + '

    ' + hit_excerpt + '
    ' if (hit_date || hit_tags) { result += '
      ' if (hit_date) { result += '
    • ' + hit_date + '
    • ' } if (hit_tags) { result += '
    • ' + hit_tags + '
    • ' } result += '
    ' } result += '
    ' results += result } $results.innerHTML = results + '

    Propulsé par l\'excellent

    ' } ;