The outer limits of scaling – how decentralized can your infrastructure be?

People PowerI’m a big Twitter fan. So it’s painful to see that Twitter is down so often. This problem with scaling from niche service to Internet starlet has plagued a good number of other young (and not so young) websites. The scaling problem itself (growing without experiencing significant performance issues) goes right back to the earliest Internet startups – if you can’t keep adding servers to meet the demand of the people clamouring to get on your site, you’re screwed.

One of the most effective ways of improving your startups’ ability to scale is to take advantage of the thing web servers do really well – serve static web pages. You gain additional benefit from the effects of local caching, so even less traffic comes through your servers. Blogger used to take advantage of this infrastructure before they integrated with Google.

An alternative is the growing market for on-demand computing resources, where you don’t buy up your own servers at all, you just rent cycles from an abstract “cloud”. Amazon made this field famous with their “Elastic Computing Cloud” (or EC2), although there are numerous companies offering similar services, such as MediaTemple, Oracle and IBM (oh, and maybe Sony). Google’s data centres follow the same pattern and are filled with hundreds of thousands of cheap pc’s, networked together into one big machine.

The rise of Software as a Service (SaaS) has meant that providers of web-based services are building their businesses around a guarantee that their service will be up, and will be performant. Whilst wholesale adoption of SaaS in companies is impeded by justifiable concerns about surrendering so much control over your server estate into the hands of a third-party, the analogy goes that you don’t keep your money under your pillow out of fear that the bank will be robbed. The upside of relying for your software services on large, dedicated third-parties, means that the future of mashups is bright: the aggregation point that your customers visit, where data is pushed between services, can be made scalable much more easily than if you had to build the entire, integrated system.

How easy can this become? If you are a company building a new online service, to what extent can you rely entirely on other companies to host your business logic and complexity? To what extent can you reduce the load on the servers you pay for down to the holy grail of completely static hosting?

The problem with trying to host a mashup as static files is that you always need someone to “pump the handle” of the system – check for updates to the data, update records, etc. I’ve written about this before and described the problem as needing a mashup “nervous system“. What happens if you start to use your customers themselves as the nervous system? A visitor’s web browser has everything it needs to make the appropriate calls to the web services your business runs on, and pump whatever data around the web that needs pumping. If you move the tasks traditionally performed by servers into the hands of your customers, suddenly scaling looks like a very different problem, one that is a lot more amenable to tasks that can be performed robustly by commodity infrastructure.

Putting questions of security aside (and reader, that is a very big ask I know), the big objection to this has been that if no-one visits your site, it will never be updated. The truth is: if no-one is visiting your site, you’re going out of business.



  1. Posted March 13, 2008 at 10:32 am | Permalink

    Interesting perspective and some great points, but I have a couple of issues in connection with mash-ups.

    Reliability is inversely proportional to complexity, as suggested by the KISS principle. The mash-up by definition adds complexity, and furthermore reliance on 3rd parties with whom you likely have no formal contract or SLA. And you are in essence at the mercy of their subject to change – or worse still fail – business models.

    Would you build your house on land that had not been surveyed, and which you neither owned nor had a leasehold? Mash-ups bring many benefits but no certainty of scale, dependability or ongoing viability.

  2. Posted March 14, 2008 at 5:02 pm | Permalink


    Thanks for the comment. My view on the reliability of mashups is that you only attempt to control the things you are capable of. I’ll outsource reliability, scaling and dependability of my application to the people earning their living by providing a good level of service in those areas. If they aren’t up to speed, I can move on and change my mashup to work with someone else.

    As the demand for mashups increases, companies will start to offer SLA’s to the people that want them. I’d thought StrikeIron provided SLA’s for the services they broker, but it would appear not. IBM has a research group looking into this.

  3. Posted March 15, 2008 at 5:44 pm | Permalink

    I’m with Andrew on this one – mashups are often great, but potentially volatile.

    Aside: The SaaS/bank analogy doesn’t stand up to close examination. For example, there are gazillions of regulations for financial institutions – but if your service provider shuts down, you mostly don’t have any leverage.

  4. Posted April 8, 2008 at 12:46 pm | Permalink

    The trouble with SLAs is that they cost money. Building and providing a basic service is relatively cheap compared to the cost in operational support systems. You still need some of these to provide a basic service that can scale Etc., but I’d assert that with fixed SLAs comes much additional complexity in terms of OSS. This is like the difference between your cheap or free web hosting, and say fully managed highly-available infrastructure such as major business outsources to BT.

    Perhaps standards and data portability could help here: if it is easy to deploy logic and data to more than one location at once, and to move it around without major refactoring or transportation costs.

    Agree with Fred on the current weakness of the SaaS/bank analogy. But then I’m sure when banks first came to be there was little in the way of regulation. But it may take a number of unfortunate incidents before governments put in place effective measures to protect SaaS consumers.

    But in any case I think service mash-ups most certainly have a brighter future than their musical counterparts – which quickly became tiring and a rather embarrassing source of momentary interest.

One Trackback/Pingback

  1. […] The outer limits of scaling – how decentralized can your infrastructure be? « Jay by Jay Fresh Jon explores the lure of smart clients and dumb servers… (tags: jonlister scalability scaling infrastructure decentralisation decentralization) Posted by jermolene Filed in Uncategorized […]