I was looking for a simple way to perform a headless install of Sun/Oracle Java6 JDK on Ubuntu on EC2.  After hunting around quite a bit I found a recipe and thought I’d post it here

Edit

and add this line

From the shell

I’ve been studying NoSQL – why it exists, what motivates it, how it differs from mere sharding, etc.   And by NoSQL, I’m thinking of only the crop of “new databases” — Cassandra, MongoDB, Voldemort, etc.   It’s a complex topic.

However, it occurred to me that one way to understand aspects of the problem is to think of a (relatively) common DB architecture and ask how/why NoSQL believes it has a better answer.  So I’m working through that process on one DB topology: a sharded, MySQL db system.   Given such a system, why might I consider using a NoSQL alternative?  Thoughts on the topic are below.

Please note that these are just thoughts in one direction — I’m not trying to capture, here, answers to the arguments a NoSQL evangelist would put forward.  Neither am I accepting the NoSQL arguments.  Rather, I’m simply trying to make myself capable of verbalizing one particular NoSQL argument, independent of whether I buy the argument itself.

So, here is the summary of the argument I’ve heard to date.  There are probably better arguments and phrasings to be had.

You’re going to have to face a few realities:

1. Master DB machines go down or get sick.   Given a number of read slaves of that master, it’s non trivial to figure out which of the slaves becomes the new master and how to get the other read slaves to begin replicating from that new master at the proper point.  (Amazon’s upcoming RDS read slaves are, therefore, pretty amazing technology)
2. When master DB machines go down, you will incur a delay in setting up a new master
3. How do you add new shards and rebalance the data?
4. You need a way to know which shard to talk to.  Perhaps by implementing your own version of Consistent Hashing?
5. Caching is also your problem.  You can use off the shelf systems, say memcached, to reduce the work but its till not zero work.
6. You will most likely give up distributed joins between the various shards.

Now, you do all this in the interest of making a CAP theorem tradeoff — you give up instantaneous consistency and distributed joins to get availability and resilience to network partitioning.   You are maximizing A and P.  And it’s all worth it, provided a database is what you really need.

But suppose instead that you find you don’t do *any* joins or range queries — not even in a single shard.  Perhaps, at the extreme, you are even storing blobs of data under a key.  The database becomes little more than a transaction manager.

At this point, the new crop of NoSQL becomes attractive — Cassandra, for example — and for the following reasons:

1. Cassandra is already focused on making a CAP tradeoff, and in the same direction; it already does data replication among nodes, implements consistent hashing for reading the data, does quorum reads, etc.
2. Because of its base assumptions, Cassandra  is better at adding/removing nodes from the group — they can be added/removed without causing wholesale rebalancing and without losing data.  No single master write node causes the system to hang.  Clients are relatively isolated from concerns about mapping a read to the proper server to read from.
3. You gain more than you lose.  Yes,you give up a lot of DB capability (e.g. range queries, arbitrary joins, proper data normalization, etc.).  But since you weren’t using those anyway, what have you really lost?  Moreover, you gain simpler administration, operation, and expansion of your CAP tradeoff.
4. Caching in Cassandra is somewhat inherent and transparent
5. A key/value code stack is faster than a SQL stack, even MySQL’s stack

Once upon a time, I was very interested in computing complexity theory. I wish I could remember, now, the subtleties of the theory. But I remember enough to be interested in the purported proof that P != NP. Very interesting.

I just finished a conference call with a vendor who built and uses their own “cloud” to host applications for their customers.    There were several lessons they learned along the way.   Without comment, here are some of the lessons they mentioned:

• XEN works great for building internal clouds.   They tried VMWare, which worked, but was far too expensive
• Building an internal cloud was far more work than they thought.
• Sizing an internal cloud is tricky.  By the time you complete an internal cloud build – which will be longer than you think, the technology will have shifted significantly and become much cheaper
• They used a SAN for storage in the cloud
• The cloud has changed their business – they used to run hosted services (the ASP model), but were always suffering because every customer wanted customization.   They now deal with that via virtualization – they just run, say, customized versions of Exchange in multiple VMs
• The recession helps cloud companies – because companies don’t want to spend on IT but are willing to spend “operational” costs.

Tab sweep of items I reviewed regarding how social policies support or enable the success of wikis:

General Wiki Statistics

Examples of Wikipedia policies / social roles

The study of wikis and why and how they work

Specific Papers I found enlightening

I’ve been wrestling for a while trying to find a good technique for large scale graph processing. Some wonderful folks here at work have come up with some cool solutions based on map-reduce.

Today, however, Google sent out a tease about their in-house solution — Pregel. It will be interesting to see what the details are when they discuss it.

Current tabs

• Blog entry from Matthew Hurst exploring Google^2
• Blog entry from William Cohen, discussing the above entry and SEAL
• Google^2
• SEAL

I’m swapping into working memory the history and relationships of URLs, URNs, and URIs….

• IETF URI Working Group
• Tim Bray on identifiers
• And again
• Untangle URIs, URLs, and URNs

Credit to Car Talk:

I can’t begin to count the number of times that I’ve seen it (and, sadly, participated sometimes — I like to believe I’m wiser now)….

Posit the question: Do two people who don’t know what they are talking
about know more or less than one person who doesn’t know what he’s
talking about?

One person will only go so far out on a limb in his construction of
deeply hypothetical structures, and will often end with a shrug or a
raising of hands to indicate the dismissability of his particular take
on a subject. With two people, the intricacies, the gives and takes,
the wherefores and why-nots, can become a veritable pas-de-deux of breathtaking speculation, interwoven in such a way that apologies or gestures of doubt are rendered unnecessary.

John Engales, the CTO of Rackspace, has written a presentation about the typical stages of scalability undergone by a website.