By Michael Woloszynowicz

By Michael Woloszynowicz

Sunday, February 27, 2011

Lean Hiring in a Lean Startup

So you've decided to start a software company and manage it using the lean startup methodology. You've started by constructing a very basic minimum viable product such as a landing page or static html pages, and have seen a pretty good response to your idea. The time has come to test your idea further by building a functional prototype. If your programming skills are good and you're able to complete the prototype in a reasonable time frame, it's always a good option to try building it yourself. If however the scope of the project is beyond your skill level you'll have to confront the issue of hiring. Hiring in a lean startup follows many of the same principles as hiring for a regular startup but there are a few extra considerations that need to be made. Please note that the below considerations are primarily valid during the customer discovery and customer validation stages.

Outsourcing is an Option
While I generally discourage outsourcing anything but the most non-core, non-strategic projects, an MVP is actually a good candidate. As my previous post (The Minimum Viable Product Dissected) mentions, you have to assume that the first few versions of your MVP will be discarded, and so you overcome a lot of the downsides of outsourcing. You don't have to worry about the quality of the code or being dependent on a third party to maintain it, and what you want is a cheap way to create something that validates your hypothesis. Depending on the scope of your MVP - and it should be small - you should be able to get an outsourced version of the product done for $2000 - $5000 which is certainly cheaper than hiring employees to develop it in house. The advantage you gain is that you're not spending precious time looking for talent when you should be validating your idea. The challenge of hiring at this early stage is that it's difficult to find people that will believe in your idea (unless you're a web 2.0 celebrity), especially since you don't even know it's good, and that will be happy working on something which will be partially or entirely discarded. David Binetti took the outsourcing route in the early stages of Votizen and was able to make several pivots with a relatively low sunk cost (see his presentation for a good case study). If you do decide to take this route, make sure that your outsourcing company has a local project manager and that they will be available for at least a few months following the MVP release to make adjustments. As long as you're given the source code you can make many of the changes yourself, and you'll no doubt have to. Expect to make adjustments to the copy and some of the graphics and layouts as you iterate and measure your AARRR metrics. This MVP will either prove that you have a strong concept or that you must Pivot. If it turns out to be the former then you can proceed to the hiring stage and begin developing a proper version of the application.

Hire Lean
Given that you are operating as a lean startup, your workforce should be pretty lean as well. You're not working from a 500 page spec (nor should you ever), and you're not employing a waterfall methodology (nor should you ever) so it would be silly to hire 10 people and send them off to work. Although you've gotten some good validation of your idea with your early MVP, you still have to build, measure, and learn. Although you can do this with lots of people it becomes more challenging as you have to deal with the overhead of coordinating so many people as well as other reasons I'll discuss later. Generally an optimal size for a starting team is 2-3 people, one to design and two to code. An ideal situation is when the founder can fill one of these roles as well.

Hire Inquisitive Minds
As I've mentioned, at this early stage you still have a lot of learning to do so it's simply not enough to hire good designers and good programmers. You have to hire people that ask good questions. Validated learning is not something that happens at the management level - which shouldn't exist at this stage. From setting up experiments to collecting, synthesizing, and interpreting the metrics, everyone on the team has to be involved.  Since the build, measure, learn feedback loop has only one "build" stage, you want to ensure that your new hires are being utilized in full. For these reasons you need to find people that ask lots of good questions during the interview process, if they're not curious about your idea and simply nod their heads in agreement, they're not for you. There are lots of great tools out there for assisting you in collecting metrics, most of which don't require a strong statistics background, but if you find someone with these skills then all the better. It is also paramount that your design hire have a good background in usability as it's more important that your application be easy to use than beautiful. Finally, returning back to lean hiring, it should now be clear that you can't keep a team of ten engaged at every stage of the feedback loop and that your agility will drop. There's also an old saying that "too many cooks in the kitchen spoils the broth", and the same applies to metrics and validated learning.

Although the above suggestions are meant to be generic, it's important that you outline your hiring criteria after each pivot as they will continually change. These changes can be anything from design style, industry knowledge, programming language, scalability experience, years of experience, to many others. This effect is further heightened once you stop pivoting and move from customer validation to company creation to company building so the hire of today won't be the same as the hire of tomorrow.

If you liked this post please follow me on Twitter for more.

Saturday, February 19, 2011

Crowdsourcing Your Way to Average

Upon first glance crowdsourcing appears to be a great way to accomplish just about anything. After all, the old saying goes that two heads are better than one, so naturally hundreds of heads make it all the better. After some recent experiences with both CrowdSpring and 99 Designs, I would argue that crowdsourcing is only beneficial in certain circumstances and restrictions.

With the growth and success of sites like Wikipedia, Reddit, and Stack Overflow, it's clear that crowdsourcing can work really well. The key difference however between the aforementioned sites and for-profit sites like CrowdSpring, is that members of the former are driven by the desire to collectively improve the content and offering of the site, while the latter is driven by individual competition in a winner takes all scenario. What I've discovered as a similarity between both types, is that the bulk of the high quality content is submitted by a very small portion of the user base. On collective intelligence sites such as Wikipedia this is still OK as this small percentage represents a large number of users in absolute terms, so they're still able to maintain a relatively high standard of quality. On CrowdSpring or 99 Designs, the end consumer is left to steer the direction of content submissions which becomes an onerous and time consuming chore. This however is not greatest problem with for-profit sites, the greatest problem is that the intelligence is not actually collective. Rather, you get a sort of "1000 monkeys working on a 1000 typewriters" syndrome where content is not gradually refined but instead individually re-invented. I ran my competitions in a public manner so each person could see other submissions and what had scored well, but somehow this did not result in a convergence towards an ideal logo. Instead you had less creative designers submitting lower quality versions of high scoring logos while others started from scratch. Having new styles and ideas come in is great, however very few designers actually took the time to carefully read the project description so you end up spending a great deal of time re-explaining requirements and guiding them towards your desired style. Since your project is not the only one going on, many designers quickly lose interest and move onto another project if their first submission didn't hit the mark.

In addition to the relatively poor outcome of this process there are also several depressing aspects to this sort of product creation. The first is that it doesn't maximize value creation since the combined efforts of so many individuals coupled with the project award value yield a pitiful average hourly rate for all involved. Viewing many of the designer profiles we see that the average win rate is in the 2%-6% range implying that either the process is inherently flawed, or the designer is simply not very good. Either way this does not bode well for the end consumer. The other downside is that while participants in collective intelligence sites are happy contributing content, those in for profit sites seem to resent the process as it drives down rates in their industry. Can we therefore get a good result from a group of disenchanted designers?

This is not to say that for profit sites can't be successful, but I think it's time to rethink the process and the constraints within which they operate. The primary driver for my use of these sites is the challenge I've had in finding a designer whose style reflects my company's taste and direction. In fact we've tried hiring a marketing company to design one of our logos and after months or back and forth we abandoned the process. What I therefore suggest, is that these sites strive to become more of a directory for design talent where designers compete not just for money but also overall standing (e.g. Stack Overflow). What I'd like to do is start by perusing a large set of designer portfolios and choosing those designers whose past work is appealing to me and reflect my companies product and design styles. At that point I invite a small number of them to take part in the competition. They review the project requirements and if they feel the project matches their design skills, each submits a few initial ideas. At each round of submission we eliminate one designer and rate designs based on design quality, originality, and applicability which add to the designers overall rating. By using a score designers are more motivated to provide high quality content as its effects are carried beyond that single competition. By starting with a small pool of designers the chances of winning are much higher, designers are more engaged with the project, and don't have to spread themselves as thinly across a large number of projects. What we are left with is a process that weeds out low quality designers while maintaining a good effort to reward ratio for the higher performers. Further it wastes less of the clients time and allows them to work more closely with the designers to achieve the desired result. While this process has not been tested in the field, I suggest it as an alternate way of thinking about crowdsourcing in the context of commerce. We need to find and cultivate an environment that emphasizes value creation rather than one sided value capture and we can only achieve this by experimenting with different business models. For a product to achieve sustainable success it must make both the content creators and consumers happy, and as it stands I don't think either side is.

In the end we asked for a refund on one site and were left with a satisfactory, albeit not ideal result on the other. Perhaps running experiments several more times I would have a better gauge of the efficacy of these sites but I can't help but think that it's not the panacea I thought it would be. These are my experiences and I'd love to hear how you've fared using such sites and what advice you may have to make them better, so please chime back in the comments.

If you liked this post please follow me on Twitter for more.

Sunday, February 13, 2011

Practical Advice for Computer Science Students

Throughout the numerous rounds of hiring I've conducted, I often to reflect on the skills I wish many programmers had and how this relates to the university experience. When thinking back to my own university experience, many of these things were not very obvious to me, so I've put together this list of suggestions for CS students. Although the below points are geared towards students, they apply equally to experienced developers, it's never too late.

Learn about usability and design
Most programmers view usability and aesthetics as the domain of designers, and that their focus is purely engineering. While it's true that your ability to write high quality code should be a key goal, usability is hardly a something you can ignore. In order to write applications that people love to use, the user experience must be ingrained in the culture of the firm and transcend departments. Too often engineers and designers are at odds with one another since making a usable system requires more work on the part of engineers. If you accept that front-end is as important as the back-end, the dynamic of the development team will improve and you'll end up with a drastically better product. After all, if nobody wants to use your software, it doesn't matter how brilliant your back-end algorithms are. Because your own application always seems obvious to you, I recommend signing up as a tester for a site like and begin testing applications. You'll soon see the difference between well and poorly designed applications. In particular, consider what makes an application easy to use and think from a programmers perspective how much work went in to making it great. You'll quickly realize that usability takes lots of work, but it is an investment worth making. While in school take a human computer interaction course and read a few books on usability (Rosenfeld Media has lots of good resources for this). When looking for a job, try to gauge the companies dedication to usability during the interview process. If the company is not interested in usability you're unlikely to change their mind, and it is certainly more rewarding to write applications that people love than those they curse.

Study statistics and data mining
While at university I knew very few people that enjoyed statistics or data mining courses. Six or seven years ago it was easy to ignore these courses but today data is king. Companies are collecting and processing more data than ever and those developers that can make sense of all this content are in high demand. In many cases you don't have to dwell on the low level details of these courses but rather understand the tools available to you and where they apply. Today there are lots of tools such as R, SAS/STAT, and Mahout to do a lot of the work for you but you still need to know where and how to apply them.

Learn the front-end, not just the back-end
When I was at school HTML, CSS, and JavaScript were all dirty words. CS students figured the back-end would reign supreme forever and that they could leave these client side technologies to others. The world has changed quite a bit since then, and these days a programmer that's not proficient with these languages is like a one armed pianist. Demand for full-stack programmers is on the rise, particularly at smaller companies, and having a healthy understanding of all application layers makes you a better architect and more well rounded programmer. Not to mention that if you ever plan on starting your own company you'll almost certainly have to develop both the front and back-end of your applications. For more reasons and advice on getting started read Why You Nead to Learn JavaScript.

Start a side project
As a new grad you may not have a great deal of work experience so a side project is a nice way to build up your resume and learn a great deal along the way. Today a Github project carries as much weight as a lengthy resume so get your name out there an start something interesting. Work on something that interests you but simultaneously try to use technologies that are new to you. This way you can go beyond your comfort zone, maximize learning, and show that you're a passionate programmer. Programming is one of the few professions where you can create something out of nothing with nearly zero cost (with the exception of your time). This low barrier is one of the reasons many employers will expect that you've tried doing your own thing, it shows you're excited about writing applications and have the drive and capability to ship a finished product.

Learn a language that's not being taught at school
Schools typically cover only a handful of languages, C++ or Java on the OO side and Haskell or LISP on the functional programming side. These are all great languages but they are no longer the be all and end all. In an effort to accelerate development many companies have adopted less verbose languages with a richer set of tools. Two excellent examples of such languages are Ruby and Python, both of which you can't go wrong in learning. Even if the company you're applying to doesn't use these particular languages, your knowledge of them will demonstrate your passion for programming. If you decide to take my advice and start a side project, Ruby on Rails is a great choice as you'll kill two birds with one stone, and drastically shorten your development time.

Work for a startup
Most CS students (outside of the Valley) aspire to work at a big company as they figure it will show well on their resume and lead to greater future success. This may be true if you work for a company with a notoriously rigorous screening process such as Google by giving credence to your abilities, but it's not always the case. The biggest benefit you can get from working at a large company is a intimate understanding of processes, but you'll gain little in the way of creativity and breadth of knowledge. Working at a startup will challenge you more, you will have a greater opportunity to make a difference, and provide you with more job satisfaction. If you decide to take the startup route ensure that you believe in what the company is doing and that it is both interesting and innovative. If the startup is a relatively cookie-cutter business you'll gain little benefit, but if you join a cutting edge company you'll be exposed to so much more than you would be at a large company. As an added benefit you'll learn what it takes to build a successful software company which will help you if you choose to start your own company in the future. Finally, most startups require long hours and give you less job security so it's best to work for one before you have a family and mortgage to support.

Only do it if you love it
Computer science is a field where it's very easy to see which programmers love what they do and those that see it as a just another job. The number one difference between a great programmer and a mediocre one is passion and it's something that innovative firms will look for in the hiring process. If you're not excited when you write code then perhaps CS is not for you. Bear in mind that passionate programmers tend to embrace many of the above suggestions so companies may use them as proxies for gauging passion.

Four years goes by quickly so be sure to tackle these tips from the get go and set goals for every year. Universities strive to give you a well rounded experience but top companies are looking beyond the standard curriculum for something extra. Most importantly keep in mind that this sort of list changes frequently as computer science is far from a static profession. Lifelong learning is a certainty so read as much as you can, stay informed of new trends and technologies, and prepare to continually tackle a new set of challenges.

If you liked this post please follow me on Twitter for more.