By Michael Woloszynowicz

By Michael Woloszynowicz

Thursday, April 28, 2011

The Freeconomics of Dropbox - Projecting Freemium

In a previous post I covered a set of quick calculations to gauge the profit potential of Dropbox. These calculations were intended not only to gain some insight into Dropbox's financials, but also to provide a simple guide for gauging freemium profitability. Given the large degree of interest in the article I decided to write a follow up, on some other things we can do with these numbers. The model used in this example can serve as a starting guide for your own freemium projection model or to play around with the Dropbox numbers. I've you haven't yet read the previous article I urge you to do so here as it will make the example in this article much clearer.

In the first article we looked at the required paid user conversion rate that would cover the costs of free users as well as fixed costs, and make the firm profitable. In order to keep the example simple, we looked at a single snapshot in time using some rough calculations. In this article I'd like to take a more dynamic and rigorous approach and look the numbers over a span of time with the help of a financial model. To help you follow along and to let you experiment with your own numbers, the Excel model can be downloaded here (xls, xlsx).

Some Freemium Basics

Before we delve into the actual numbers for Dropbox, I should explain the rationale behind the calculations and a concept that I call convergence. With any freemium model we have a set of free users that result in a pure cost to the firm, as well as a set of paid users that bring in revenue. Since user conversions typically lag new free user signups, and the fact that there are far more free users than paid users, there's often (but not always) a long period during which the firm encounters losses. A good way to represent this is with 2 lines on a graph (as shown below), one of which is the cost of free users plus fixed costs (in blue), while the other is profitability of a paid user (in red).


Above we can see two scenarios, one where the line's diverge, and another where they converge. If the two lines are diverging then clearly they will never cross and the company will never make a profit. At the point where the lines converge, and the paid user profit line exceeds the cost line, the firm begins to make money and the freemium model starts paying off. To put it another way, if the slope of the free user cost line is smaller than the paid user profit line then the firm will be profitable at some point.

In the above graphs I use a linear relationship but it's more likely that user growth will be exponential due to network effects and referrals (as shown below). The model provided uses an exponential growth rate that can be modified to evaluate different scenarios.


In order to yield a desirable profit scenario, the are several levers that the company has available to them. The main ones, and also the ones used in our model are as follows:
  • The paid user conversion rate which is effected by the level of functionality provided to free users vs. paid users, as well as the value of your service overall
  • Maximum storage limit for free users, a lower space limit results in a lower cost
  • The price of a paid account
  • The cost of storage, transfer or any other variable or fixed costs


The Model Applied to Dropbox

So without further ado, let's dive into a more detailed look at how Dropbox is doing using our newly constructed model. I've made a few adjustments to my assumptions based on reader feedback to the previous article so they are as follows:
  • I assumed that Dropbox officially launched in September of 2008 at TechCrunch 50 so when looking at the spreadsheet, note that the current date is at about week 140.
  • Several users felt that an average upload size of ~1.6M was too large and suggested that about 50% of that was more realistic. In this case we calculate that the average user uploads 8MB per day (10 files * 0.8MB) and downloads the same amount. The model factors in the storage cap of 2GB for free users and 50GB for paid users. 
  • Since the new model is no longer static, I've assumed a starting fixed cost of $8000 per month with a growth of $2400 per month. This growth rate gives us the $367,000 per month rate that Dropbox is at now (as per the previous article). This fixed cost primarily includes employee salaries and was calculated in the previous post. 
  • Our cost of storage, transfer, EC2 instances, and requests are all based on Amazon's published rates and have not changed from the previous post. 
  • I've chosen an initial signup rate of 1,000 users per week with an exponential growth rate of 1.2. This gives us a good approximation of user growth as it lines up with the 3M users in the GigaOM in 2009, and the 25M user mark today. 
  • It's assumed that free users store an additional 40MB every month until they reach their 2GB cap, while paid users store 12.5x this much (note 1). This figure gets us to our 433MB/user average that we calculated in the previous post, based on the GigaOM article. 
  • Projections are made over an 8 year period as some interesting results show up in later years
  • Paid user drop-off is ignored for reasons explained in Note 3
Note that all assumptions are shown in the Excel model (note 2) and can be modified to your liking to see how scenarios change. Below I provide a few of the more interesting scenarios I've found. Please note that the figures presented are on a weekly basis. 

Scenario 1

Under this scenario we assume a paid user conversion rate of 3% and that a paid user does about 12.5x more downloading and uploading than a free user (note 1). This produces the following graph:


What we see from the above graph is that Dropbox is currently (at week 140) profitable, earning about $1.8M per month on costs of $3.7M. However; an important thing to note is that as free users gradually consume more of their space, they will cease to be profitable in week 295 as costs will rise more rapidly than paid user revenue. Two possible solutions to this would be to cap accounts at 1GB, or improve conversions, as we'll see next.

Scenario 2

Under this scenario we assume a paid user conversion rate of 5% and that a paid user does about 12.5x more downloading and uploading than a free one (note 1). This produces the following graph:


At a 5% paid user conversion we see that DropBox is a runaway hit, currently (week 140) earning about $5.4M per month. This time the profitability will continue as the slope of the tangent for the paid user line at the end of year 8 exceeds that of the free user cost line. It turns out that at a 4.6% conversion rate, the slope of both tangents will be more or less equal and Dropbox can sustain profitability over the long term. If Dropbox were to cap free user storage at 1GB then a 4% conversion would suffice for sustained profitability. 

Scenario 3

Under our third scenario we introduce deduplication. It was suggested by some readers that a deduplication gain of 20% is unlikely so let's reduce this to a more conservative 10%. In doing so we result in the following graph at a 3% conversion rate (as in Scenario 1). 


We see that at 3%, even with deduplication we still get an undesirable convergance by the end of year 8 so we'll need a higher conversion rate. Despite this fact, deduplication has resulted in a profit today of $3.1M per month, a $1.3M increase over not using deduplication. It is easy to see now why they chose to use it. 

Scenario 4

In our final scenario we revisit the 5% conversion rate from scenario 2 but with a deduplication of 10% factored in. 


It is clear that at 5% with deduplication, Dropbox is making money hand over fist. Profitability today is $6.8M per month, and over $40M per month by the end of year 8. It turns out that thanks to deduplication, Dropbox needs a conversion rate of about 3.7% to be sustainably profitable. 

Summary

There are several takeaways that we can infer from the model and above scenarios:
  1. Dropbox has tremendous potential provided it can achieve the necessary conversion rate of 3.5-4%
  2. Dropbox could easily have been profitable at a very early stage and likely is today
  3. Freemium models need to be studied over an extended period of time rather than as snapshot in time
  4. Our back of the envelope estimates in part 1 were not too far off but they also didn't show the full picture
  5. With some discounted cash flow analysis it's easy to see that under a 5% conversion rate, with deduplication, Dropbox could be worth well north of $1B, but this is a discussion for another post. 
If you've not already done so, I suggest you download the Excel model (note 2) that I used for these calculations and play with the numbers on the assumptions sheet. Feel free to tweak it as needed it or use it as a starting point for formulating your own freemium models. 

If you liked this post, please upvote it on Hacker News, Tweet it, or follow me on Twitter

Note 1 - This is based on our average case from the previous article. A paid user has 25x more space than a free user, hence the midway point of 12.5x. 
Note 2 - The spreadsheet is available in both .xls and .xlsx formats. The model includes several sheets but focus your attention on the "Assumptions" and "Financials" sheets. Since it took a fair bit of time to create this spreadsheet I ask that you please note the original author if you reference it. 
Note 3 - Paid user drop-off was not explicitly incorporated as it exists for both paid and free users. Since both types of users drop-off it would be factored in by the user growth rate and the conversion rate. Drop-off can easily be incorporated into the model if you have specific numbers.  

Wednesday, April 20, 2011

The Economics of Dropbox

Dropbox's recent 25 million user milestone got me thinking about the economics behind the company and what it would take for them to be profitable. Given that Dropbox uses Amazon Web Services as their hosting platform, along with S3 for document storage, some rough cost figures are fairly easy to come by. After all, we know the company has 25m users and each free user gets 2 GB worth of data. The trick however is S3's charges are based on used data, not committed data so the big question mark is how much space does the average free user consume. This can be quite a large spread as my free account actually has 4 GB (thanks to many referrals), with about 2.5 GB used, while others I know use only ~10% of their 2GB account. Similarly we know that Dropbox uses deduplication to prevent duplicate storage of files and of course its effectiveness depends on the uniqueness of the average file being uploaded. If a good number of free users use their account for storing torrent downloads e.g. movies, songs, etc, then the amount of storage used can be reduced drastically. A Quora post I found suggests that the savings for deduplication are in the 5%-20% range. Given the large number of users and high likelihood of duplication for media files, let's take 20% as our estimate. Finally, any shared data is stored only once despite being shared across 2 or more users, so we'll roll this into our deduplication estimate of 20%. Given that specific data is not published, all we can really do is speculate and formulate a cost range. For starters let's assume that every user is a free user with 1.6 GB committed (2 GB - 20% deduplication savings), this will serve as our high cost. A 2009 post from GigaOM indicates that at the time, Dropbox had 3M users and the company was storing 1.3M GB of data. If we assume this proportion has continued we get a per user storage of 433MB, we'll use this as our low number.

Tier (GB) S3 Cost/GB/Month S3 Cost/Month
1000 $0.14 $140.00
50,000 $0.125 $6,125.00
500,000 $0.11 $49,500.00
1,000,000 $0.095 $47,500.00
5,000,000 $0.08 $320,000.00
40,000,000 $0.055 $1,925,000.00


$2,348,265.00

Above we see the calculations for each Amazon S3 pricing tier for our high cost figure, yielding a storage cost of just under $2.4M/month which represents the largest cost to Dropbox. Performing a similar calculation with our low estimate, the number would reduce substantially to $640K per month. I expect that the true number will be somewhere in this range so we'll consider these our high and low values.

In addition to storage we have to factor in S3's per request cost. I assumed that the average user will make 10 put/copy/list requests and 10 get requests daily, and given amazon's flat fee of 1 cent/1000 put/copy/list requests, and 1 cent/10000 get requests, we get a cost of $82,500 per month.

Dropbox claims to handle nearly 200M uploads per day and I've estimated the average stored file at 1.6MB based on my own account as well as those of a few people I've asked. Once again deduplication becomes a factor so we'll take another 20% correction, resulting in a daily upload of 25.8 TB's. At a rate of 10 cents/GB we get a cost of $770,000/month. Since download numbers are unpublished, we can only guess that it's close to the upload number since some files are downloaded multiple times (on a few devices), while others are merely backed up. With Amazon's rate of 8 cents/GB we get around $620,000/month. The combined transfer cost is going to be around $1.4M/month.

Of course Dropbox also needs a good amount of servers to handle all the heavy lifting. With a a mix of SimpleDB, RDS, and web servers, and given their volume of requests and number of users, it's easy to imagine that at least 200 instances are needed. We'll treat each server as a middle of the road on-demand server at a cost of 30 cents/per hour which yields a cost of $43,000 per month.

Finally, a number that's quite easy to estimate is their payroll. Dropbox's LinkedIn page lists 44 employees, and with an average Valley salary of $100k/year we come up with a monthly cost of $367,000.

Adding all this up gives us a cost in the range of $2.5M - $4.4M per month. So what can we infer from these numbers? First off, working with these same assumptions (less payroll costs) we can determine that a single full 2GB free account costs the company around 16 cents/month, while our low case account (with 433MB used) costs about 9 cents/month (see note 1). The most important thing to consider is how many paid Pro 50 accounts they would need to cover their costs. To get this number we need to figure out how profitable a paid user is. This once again depends on the amount of data the paid user has stored, but we can certainly expect that it's over 2GB, otherwise they'd use a free account. Since we've calculated the cost of a 2GB account to be around 16 cents/month, let's start by assuming that paid users are using only 2GB and therefore each one results in a $9.86/month profit (see note 2). At this level of profitability we see that Dropbox would need between 254,000 and 450,000 paid users to cover costs, or a conversion rate of 1% and 1.8% respectively. We could imagine a worst case profitability scenario being a paid user that consumes the same amount of resources as 25 users (50GB is 25 times as much storage as a free user). The cost of such a heavy user is $4.15 (see note 3) leaving $5.85 as profit. At this much lower profitability level, Dropbox will need between 427,000 and 750,000 paid users, or a conversion rate of 1.7% and 3% respectively. Running this calculation at a paid usage somewhere in the middle results in a required conversion rate of 1.3% to 2.2% (see note 4).

My guess is that the low usage numbers for free users, and average usage number for paid users are the closest to reality. With that in mind it looks like Dropbox needs a conversion rate somewhere in the 1% to 1.5% range. Since freemium conversion rates run anywhere from 0.5% to 5% this is neither a low nor high hurdle, and is certainly attainable. Dropbox's challenge is that these numbers will constantly shift as they depend heavily on the speed at which free users convert vs. how quickly new free users sign up and consume their space. If the rate of new sign-ups drastically outpaces conversions, then Dropbox will need to seek further venture financing. The final thing to consider is that Dropbox's total funding is in the $7.2M - $10M range. Given the high burn rate of $2.5M-$4.4M per month, Dropbox must have a good base of paid users already as they wouldn't be able to survive on outside financing alone.

Part 2 of this article is now available and offers a complete Excel model of Dropbox that you can download.

If you liked this post please follow me on Twitter for more.


Please note that the above numbers are "back of the envelope" style calculations and are not meant to be perfectly accurate. The intention is to gain some insight on how Dropbox's costs might break down and the profitability potential the company has.

Note 1 - These numbers are primarily driven by the marginal cost of the S3 storage as listed in the grid above. Transfer costs are expected to be similar in both cases as most users will fill up their account over time so more storage use doesn't necessarily imply more daily transfer use. For the full 2GB's of usage we get 18 cents as the storage cost plus 7 cents for data transfers. I've ignored the EC2 cost as it becomes nothing more than a rounding error at a per user level. 
Note 2 - We get this profit by taking the $9.99 price minus the 16 cent cost for 2GB plus we add back the 3 cents of storage cost for our average 433MB user. 
Note 3 - Since we've assumed the data transfer cost of a normal user to be around 7 cents per month (as used in Note 2), the data cost for our heavy paid user is 25*0.07 or $1.75. The cost of 50GB of S3 storage is $2.40 so the total cost is $4.15. 
Note 4 - The user will have used 25GB of data at a cost of $1.20 and used 12.5 times more data transfer than the average user at a cost of 87.50 cents yielding a total cost of $2.07. 

Saturday, April 16, 2011

You Don't Know JavaScript

Over the last year or so I've noticed an irritating phenomenon developing. I've seen a repetitive pattern of programmers dressing their resumes with technologies that they don't really know, but have merely touched on. While this seems to be happening with many languages, the most commonly violated language is JavaScript.


You Don't Know That You Don't Know It
The reason for this is that just about every web developer comes across the need for JavaScript at one point or another. With a lack of understanding, the most common approach to learning JavaScript is to search for code samples case by case and perform some quick copy/paste. The problem with this sort of 'learning' is that developer never actually learns the language, and instead gains the false sense that they know it. What I've discovered over the course of learning and working with JavaScript for several years, is that you don't know that you don't know it, until you actually know it. Since this is a bit of a circularity, what you really need is someone to tell you that you don't know it, and that some real learning is needed. Too often I interview someone that proudly lists JavaScript on their resume having only done some simple onClick handlers or form validation that they pieced together from code samples. The use and knowledge of frameworks such as jQuery or Dojo is great, but you can't become a master of these toolkits without a good understanding of the JavaScript behind them. In order to present the many elements of JavaScript, I have divided its concepts into what I feel are basic, intermediate, and advanced knowledge levels as follows:

A basic level of understanding for JavaScript includes:
  • Knowing the syntax of basic programming tools such as loops, if statements, try/catch, etc.
  • Understanding function definitions including the various ways they can be defined and assigned, as well as anonymous functions
  • Understanding basic scope principles, global (window) scope versus object scope (closures excluded)
  • Understanding the role of context and the use of the 'this' variable
  • Understanding the different ways to instantiate and declare an object as well as functions as objects
  • Understanding JavaScript comparison operators like '<', '>', '==', '===',  what is falsy, and how object and string comparison works, as well as casting
  • Array indexing for object attributes and functions and how this differs from actual arrays (object literals vs. array literals)
An intermediate level of understanding includes:
  • Understanding timers, how they work, and when/how they can be useful as well as asynchronous method execution
  • In depth knowledge on callbacks and function application such as the 'call' and 'apply' methods for controlling context and function argument passing
  • Understanding JSON notation and the 'eval' function
  • Understanding closures, how they affect the performance of your code, and how they can be used to create private variables, along with the lovely (function(){})() call
  • AJAX and object serialization
An advanced level of understanding includes:
  • Understanding a methods 'arguments' variable and how it can be used to overload functions through arguments.length and make recursive calls through arguments.callee. It should be noted that use of arguments.callee can be dangerous as ECMAScript 5 Strict Mode doesn't support it, although both jQuery (up to 1.4) and Dojo take advantage of it. 
  • Advanced closures such as self-memoizing functions, currying, and partially applied functions
  • Function and html prototyping, the prototype chain, and how to use base JavaScript objects and functions (e.g. Array) to minimize coding
  • Object type and the use of instanceof and typeof
  • Regular expressions and expression compiling
  • With statements and why you shouldn't use them
  • The most difficult part of all, knowing how to tie all these tools together into clean, robust, fast, maintainable, and cross browser compatible code. 
The final point in the advanced section is particularly important, and is also the hardest to achieve. Given the lax nature of JavaScript, it's easy for your application to spiral into a mess of unmaintainable spaghetti code. Once you learn the JavaScript language itself, the real mastery comes from being able to structure it and tie it together in the context of a large web application. It is this final point that requires years of practice and getting it wrong, and can't be learned from a book. I myself have been using JavaScript for several hours daily for several years, and continue to discover better ways of structuring and writing my code. It is for this reason that jumping straight into a framework is dangerous, as jQuery code has a habit of becoming unmaintainable. Dojo offers some help with this through its Class and Package system. 

Given that JavaScript has now permeated into the back-end with things like Node.js, I've decided to isolate the above requirements from web-specific knowledge. It is the web aspects (namely the DOM and IE) that have given JavaScript a bad name and send chills down every programmers spine. That being said, if you're looking to use JavaScript in a web context there are additional things that every good developer should know:
  • The DOM and manipulating it in an efficient manner, namely adding, removing, and changing nodes, as well as working with text nodes. This includes minimizing browser re-flows through the use of tools such as document fragments. 
  • Extracting information from DOM elements in a cross browser manner (e.g. style, position, etc.). Things such as this are best done with a framework like jQuery or Dojo, however, it's important to understand the differences between extracting information specified in CSS vs. style tags and computing positions and sizes, etc. 
  • Cross browser event handling, binding, and unbinding, bubbling, and how to achieve desired callback context. Again this is best handled via a framework but one should understand the differences between IE and W3C standard browsers. 
  • Expandos vs. attribute setting, the performance differences between them, and the naming discrepancies that exist
  • Regular expressions for DOM node extraction
  • Effective browser functionality detection and graceful degradation
As you can see from the above list, there is a hell of a lot more to JavaScript than alert(myVal) and myBtn.onclick = ... There is certainly a lot more to it than you can copy/paste from, and you can only become a true JavaScript programmer through reading and practice. Two great books for covering all these topics are JavaScript: The Good Parts, and Secrets of the JavaScript Ninja. Let's remember that JavaScript is quite possibly the most accessible language there is, everyone has a browser, and there is little to no setup time. Create a simple html page and start playing around with the above concepts. As for the resume decoration, I would say that if you've covered the beginner level and are venturing into the intermediate stages, it is justifiable to list JavaScript. Once you find yourself developing your desired functions rather than copying and pasting them, you can then claim to know JavaScript, until then, please don't advertise it. 

If there are any aspects of JavaScript that I've missed please chime in, in the comments section. Also please share any experience you've had with people claiming to know JS or other languages.

It should be noted that I am not a front-end developer, I'm a back end developer that has evolved into a full-stack developer. Today almost every back-end developer needs to learn JavaScript and that is what this article is meant to encourage. It's not meant to sound condescending as I hardly claim to know everything there is to JS. What I would like to see is more people realize that JavaScript is a vast and powerful language, and that there is more to it than initially meets the eye. 

If you liked this post please follow me on Twitter for more.

Sunday, April 10, 2011

Dojo Package Loading Hacks and Best Practices

One of key strengths of the Dojo Toolkit is its class system and package loader. It allows you to neatly structure your code and load classes on demand when they are needed. While the package loader is fairly straightforward on the surface, there are a few nuances that are worth exploring. It's also important to understand how best to structure your classes and when to load them so that you optimize not only the load time, but also the users perception of load time. This article assumes you understand the basics of the dojo.provide and dojo.declare functions as its focus is the dojo.require function which does the actual loading. If you're not too familiar with them, I refer you to the this article from DojoCampus before you proceed any further.

A sample class that we will use throughout this article
//Register our packages with the class loader in some other file
dojo.registerModulePath("my", "../../scripts"); 
//Tell the class loader that this class exists and will be declared
dojo.provide("my.loader.Example"); 
dojo.require("my.loader.firstlevel.Dependancy"); //Global level require
dojo.require("my.loader.Base"); //A base class the class will extend from
//Declare the actual class and extend a base class
dojo.declare("my.loader.Example", [my.loader.Base], { 
  someArray: [ ] //Don't ever do this! Treated almost like a static variable

  //The constructor
  constructor: function(args) { 
    this.helper1 = new my.loader.firstlevel.Dependancy();
    this.someArray = [ ]; //This is the right way to initialize a class level object/array
    this.loadRequirements();
  },

  //Load in all the requirements we need throughout our class
  loadRequirements: function() {
    dojo.require("my.loader.secondlevel.MyFirst");
    dojo.require("my.loader.secondlevel.MySecond");
  },


  //A method that does an on demand load
  doSomething: function() {
    dojo.require("my.loader.secondlevel.SecondDependancy");
    var d = new my.loader.secondlevel.SecondDependancy();
    return d.doSomething();
  }
});
As you can see above, we have a simple class that extends a base class, and has required  the Dependancy class that it instantiates in its constructor. To help understand this example let's discuss the mechanics of the require function.

The Require Function
The dojo.require function takes a string argument that is the canonical class name of the desired class. In the case of our Dependancy class, Dojo assumes the class will be located in [some_root]/my/loader/firstlevel/Dependancy.js and if it's not then the require will throw an exception. If you are loading in native Dojo classes that's all that is needed to make a class available for use. If however, you are requiring a class you've written that is located outside the dojo root folder, you must use the dojo.registerModulePath method we've used above. This tells the package loader where to find your package, thus defining the [some_root] value in our path above. It's generally best to call registerModulePath in a template file to ensure that it's added to every page of your web application. Once the JS file has been downloaded, Dojo will make any classes specified by dojo.provide available for instantiation.

The require function is synchronous provided it's being loaded via the normal loader. If however, Dojo was created with an xdomain loader and is housed in a module path that is a whole URL, Dojo will make a cross-domain load that is asynchronous. This obviously has important implications as you can no longer assume that a class will be available immediately after a call to dojo.require, instead you'll have to use dojo.addOnLoad to ensure it has loaded. For the remainder of the article we'll assume you're using the normal loader as this is the most common case.

Finally it should be noted that multiple calls to dojo.require do not hurt performance. Dojo knows when it has loaded a package and will therefore not make redundant requests for the required file.

The Four Levels of Require
There are effectively four levels at which you can require your classes: the global level, the package level, the class level, and the on demand level.

Requiring classes at the global level loads in all class dependencies specified in dojo.require as soon as the page loads, this naturally has implications for the load time of your page. If you make a series of require calls in the header of your page, the page content won't load until all required modules, and their package level modules have loaded. The typical scenario is to require whichever classes are going to be instantiated within the page and then relying on the package level loading to ensure all dependencies are made available to those classes.

Requiring classes at the package level - as seen in our Dependancy class above - causes the class to be downloaded as soon as the JS file/package containing that require statement is loaded via a call to dojo.require. In our example above, if our global scope were to make a call to dojo.require("my.loader.Example") it would first download /my/loader/Example.js and in turn download /my/loader/Base.js and /my/loader/firstlevel/Dependancy.js assuming these are the files the classes are contained in, but more on that later. As mentioned above it should be up to the specific package (or file) to ensure that it makes dojo.require calls for every class that it depends on. Failing to do so and relying on global level requires can cause problems when your class is reused in other contexts.

Requiring classes at the class level can be thought of as loading in required classes once a class is instantiated. In our above example this happens in both the constructor and via the call to loadRequirements. Class level loading is useful when you want to defer file download until the actual class is instantiated, but when you want every dependency to be available throughout the class. For example, if your class yields a wizard style page that uses different Dojo widgets at each step of the wizard, you can choose to load every possible widget at the class level, or you can load just the widgets that are needed for the currently shown wizard page (on demand loading). The former results in quick rendering of each wizard page since the widgets are already downloaded, while the latter results in a faster initial page load, but slower page switching since the unloaded widgets must first be downloaded.

Requiring classes on demand simply causes packages to downloaded when they are needed. We described this scenario above when discussing the multi-step widget example. Once again with on demand loading you are deferring the load time to a later point.

So When Do I Require It?
There is no one size fits all solution as to when packages should be loaded but there are a few tips that apply in a majority of cases.
  • At the global level (or window level), only require the classes/packages that you plan to instantiate at the global level
  • At the package level, only require the classes/packages that your declared classes extend from
  • At the class level, only require all dependencies upon construction if wish to ensure optimum response time during use, otherwise defer loading the package until it is used (on demand)
  • Use on demand loading if you are willing to deal with a slower response time at the time of use. For example a button click may trigger a package download before some action takes place. If this is the case be sure to provide loading feedback to the user so they understand that a delay exists. 
  • Use on demand loading for components that may not execute, thus never need to be loaded
Since the global level and package level loading is clearly defined, the real trade-off exists between class level loading and on-demand loading. When you look at this more carefully it actually becomes a usability issue as it affects the speed, or perceived speed of your application. 

Complex pages can result in an immense amount of packages being downloaded as they may use a large number of widgets, each of which has quite a few dependencies of its own. Pre-loading these widgets at the class level may seem silly since it will delay the initial load of your application. If however, it is guaranteed that the end user will end up loading each widget throughout the life cycle of the page, it's often a good decision as it will make the users experience on that page much smoother. This strategy only works on pages that are used infrequently, as users will be more willing to tolerate an initial wait. If you have a page that users access frequently, it's best to delay loading for as long as possible to improve the initial display of the page. Under both scenarios, if you choose to bulk load a number dependencies, it's wise to provide a progress bar. The movement and feedback of the progress bar gives the end user an impression that the load is quicker than it actually is. Although it's nice to add this, it's not as simple to implement as one might expect. Even adding an animated gif is non-trivial as the single-threaded nature of JavaScript causes the animation to freeze during calls to dojo.require. To help you implement this in a cross-browser manner I've provided a template class below. 

Loading packages with a progress bar
dojo.provide("my.loader.Example"); 
dojo.declare("my.loader.Example", null, { 

  //The constructor
  constructor: function(args) {
    var prog = addInitialUI(args.rootDomNode); 
    this.loadRequirements(prog, function() {
      prog.destroyRecursive();
      dojo.empty(args.rootDomNode);
      this.addActualUI(this.rootDomNode);
    });
  },


  //This builds the rest of the content of your page
  addActualUI: function(/*DOMNode*/ insertIn) { //Do Something },

  //Build our progress bar
  addInitialUI: function(/*DOMNode*/ insertIn) {
    //We need to at least load this now
    dojo.require("dijit.ProgressBar");
    var progressHolder = dojo.create('div', {}, insertIn);
    var progress = new dijit.ProgressBar({ style: { width: "300px" } }, progressHolder);
    return progress;
  },

  //Load in all the requirements we need throughout our class
  loadRequirements: function(progressBar, onFinished) {
    var rqs = [ ];
    rqs.push("my.loader.samples.Package1");
    rqs.push("my.loader.samples.Package2");
    rqs.push("my.loader.samples.Package3");
    rqs.push("my.loader.samples.Package4");
    //etc.
    var load = function(pos) {
      if (pos < rqs.length) {
        progressBar.update({ maximum: items.length, progress: pos });
        dojo.require(rqs[pos]);
        //IE won't render the update to the progress bar without a small delay
        dojo.isIE?setTimeout(function() { load.call(this, pos+1), 10):load.call(this,pos+1);
        /*
          To work with asynchronous loading we could do something like this
          instead of the above line
          dojo.addOnLoad(function() {
            dojo.isIE?setTimeout(function() { load.call(this, pos+1), 10):
                load.call(this,pos+1);
          });
        */
      }
      else onFinished.call(this);
    };
    load.call(this, 0);
  }
});

Anything Else I Should Know?
It's important to remember that the bulk of the load time is spent not on the size of the individual elements (to a point), but rather on the number of elements loaded. As a result, when writing your classes it's wise to group like classes or tightly coupled classes together into one file to reduce load time. A good example of this is Dojo's tree class which groups the Tree and TreeNode classes together into one file, since a TreeNode is never instantiated outside of the Tree. If you decide do this, classes like TreeNode should be preceded with and "_", e.g. _TreeNode, to indicate that they are private and shouldn't be constructed. While it's not ideal from a structure standpoint, grouping like elements together into one file is an option when load time is paramount. For example if you create a Tree class and decide to extend it into a CheckBoxTree class, requiring the CheckBoxTree class will result in two requests, one to Tree.js and another to CheckBoxTree.js. Another option is to include the CheckBoxTree class together with the Tree class in a file called Tree.js. What's important to remember is that when you want to use CheckBoxTree you must call dojo.require("my.Tree"), not dojo.require("my.CheckBoxTree") since Dojo will not find the /my/CheckBoxTree.js file. CheckBoxTree will automatically be made available when you require my.Tree since the provide("my.CheckBoxTree") line will be called upon loading the Tree.js file. Both of these variations are shown below. A better solution to manually grouping similar classes is to use Dojo's build system. Although it involves a bit of setup it's extremely useful for pages that load in a large number of dependencies. More information on it can be found here. If you simply want to group a number of Dojo classes together, Dojo now provides an easy to use, web-based version of the builder that will produce a single minified file in a matter of seconds. Using the builder can drastically reduce the number of individual server requests being made so it's definitely worth exploring.

Grouping Tightly Coupled Classes
//We cannot reverse the order of these declarations
dojo.provide("my._ListNode");
dojo.declare("my._ListNode", null, {
   //Class body
});

dojo.provide("my.List");
dojo.require("my._ListNode"); //Not necessary but good practice
dojo.declare("my.List", [my._ListNode], {
  //Class body
});

Grouping Similar Classes
dojo.provide("my.List");
dojo.declare("my.List", null, {
   //Class body
});

dojo.provide("my.Vector");
dojo.require("my.List"); //Not necessary but good practice
dojo.declare("my.Vector", [my.List], {
  //Class body
});

dojo.provide("my.LinkedList");
dojo.require("my.List"); //Not necessary but good practice
dojo.declare("my.LinkedList", [my.List], {
  //Class body
});

//This function would exist somewhere else
function createAList() {
  dojo.require("my.List");
  var ll = new my.LinkedList();
}

Hopefully this post has given you some more insight on Dojo package loading and the options you have. Remember that your goal is not always minimizing load time, it's also about minimizing the impression of load time.

If you liked this post please follow me on Twitter for more.