Massive-scale Online Software Development

Cutting right to the chase: Would it be possible to create software, entirely developed and moderated by an open community?

Call it democratic software development, or open source on steroids if you will. While discussing this the default answer I usually get is “it can’t be done”, which is why I gladly filed this post under the newly created category “Crazy Ideas”. Nevertheless, I find it a valuable exercise to discuss any nutcase ideas, in order to evaluate how far-fetched they actually are.

A person with a new idea is a crank until the idea succeeds. – Mark Twain

So what would such a system look like? What would be some of the requirements?

Easily accessible.
And when I say easy, I mean it. You shouldn’t have to download the repository first. You shouldn’t have to set up a development environment. It should run as a web service on the cloud (more buzzwords coming up!). A user account and an internet connection is all you need to get going.

Motivate people to participate.
Ever heard of gamification? “Gamification is the use of game design techniques and mechanics to solve problems and engage audiences.” If you are a software developer, chances are you ended up on Stack Overflow at some point. It’s a Q&A site for programmers which is quickly becoming one of the main resources for help for professional programmers. Stack Overflow incorporates many aspects of gamification, and it’s mere existence shows the power of it. A significant amount of developers is prepared to share and learn in this fun environment. Quality content is pushed to the top via a voting system, while erroneous posts are addressed by the community.

Divide work in small enough tasks.
The key to dividing work across many people is to divide it in such a way that any person only has to implement one small aspect of it at a time. Traditional software development where somebody develops a feature from a to z won’t work. One programming paradigm which at first sight seems extremely suitable for this is functional programming. A person could implement functions, and define new functions on which he relies. Combining this with aspects from test driven development where the caller has to comment and write tests for the desired function would result in automated testing.

Without worrying about the specifics too much (it’s just a nutcase idea after all) consider what the possibilities would be. In Luis von Ahn’s great TED talk the CAPTCHA inventor discusses how he re-purposed CAPTCHA in order to digitize books. Around 2,5 million books a year can be digitized through this massive-scale collaborative effort. Their next project indicates this doesn’t have to be limited to really mundane tasks. They are now working on translating the web!

Moderation guided by conventions.
Conventions are important in a group effort. Unfortunately, when discussing programming conventions people most often discuss naming and formatting conventions, while there are plenty of other important conventions to agree on. This will most likely be the topic of one of my future posts. Conventions should be as unambiguous as possible in order to know where to expect a certain piece of code, or where to place it. Conventions like these could be agreed upon through a democratic process, which seems to be working pretty well for Stack Overflow through its meta site. This allows for community moderation following the guidelines established by the community.

Couple all the separate work together into one entity.
Going from a set of loosely coupled functions to a working library would result in plenty of extra challenges, but also opportunities. Since nobody wants an all encompassing library just to use part of its functionality, the system should allow you to extract just the functionality you are interested in.

Beyond the idea

Well, … I went a bit further and attempted to start a small proof of concept. I figured the Stack Exchange platform on which Stack Overflow runs already encompasses much of the desired functionality, and creating a small scale library on it would be possible.  The idea was to create an extension method library for C#, which exists primarily out of a set of functions. Requesting new Stack Exchange sites is possible through Area 51. Not unexpectedly, my idea got shot down since it doesn’t fit the intended Q&A format. Oh well, … one can only try.


At the UIST 2014 conference, Thomas D. LaToza presented “Microtask Programming: Building Software with a Crowd”a system encompassing many of these ideas and actually evaluating them, resulting in usable code. The paper is available on ACM.

  1. #1 by Anand Jeyahar on March 4, 2012 - 7:32 pm

    It would be interesting to see someone volunteering the server instances for it.. Though i think the administration of such an instance can easily become a night mare.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: