I just finished a conference call with a vendor who built and uses their own “cloud” to host applications for their customers. There were several lessons they learned along the way. Without comment, here are some of the lessons they mentioned:
XEN works great for building internal clouds. They tried VMWare, which worked, but was far too expensive
Building an internal cloud was far more work than they thought.
Sizing an internal cloud is tricky. By the time you complete an internal cloud build – which will be longer than you think, the technology will have shifted significantly and become much cheaper
They used a SAN for storage in the cloud
The cloud has changed their business – they used to run hosted services (the ASP model), but were always suffering because every customer wanted customization. They now deal with that via virtualization – they just run, say, customized versions of Exchange in multiple VMs
The recession helps cloud companies – because companies don’t want to spend on IT but are willing to spend “operational” costs.
I’ve been wrestling for a while trying to find a good technique for large scale graph processing. Some wonderful folks here at work have come up with some cool solutions based on map-reduce.
Today, however, Google sent out a tease about their in-house solution — Pregel. It will be interesting to see what the details are when they discuss it.
I can’t begin to count the number of times that I’ve seen it (and, sadly, participated sometimes — I like to believe I’m wiser now)….
Posit the question: Do two people who don’t know what they are talking
about know more or less than one person who doesn’t know what he’s
talking about?
One person will only go so far out on a limb in his construction of
deeply hypothetical structures, and will often end with a shrug or a
raising of hands to indicate the dismissability of his particular take
on a subject. With two people, the intricacies, the gives and takes,
the wherefores and why-nots, can become a veritable pas-de-deux of breathtaking speculation, interwoven in such a way that apologies or gestures of doubt are rendered unnecessary.
By it’s own description, “Scalr is a fully redundant, self-curing and self-scaling hosting environment utilizing Amazon’s EC2″. While I haven’t yet tried it, there certainly seems to be a market for management tools riding on top of EC2. Rightscale.com, a commercial alternative to Scalr, can attest to that.
I find the type of automation offered fairly compelling. We are approaching the day where deploying your application at internet scale will be push-button simple. Perhap’s Time’s person of the year this time ’round should be a cloud.
While spending some time with Bloom Filters, I came across an interesting hashing technique called Cuckoo Hashing. In short, Cuckoo Hashing is a technique for building a hashtable with guaranteed O(1) access time. Very useful. Unfortunately, after poking around the net a bit, I wasn’t able to find any standalone implementations in Java. So I wrote one. If you find it useful, please drop me a line.