First, ambition has negative connotations from my middle-class perspective (it's blind, it's selfish). I might say D&D is about Progress, or Mastery. You start out being crappy at everything and with luck, smart decisions, and perseverence you get better at things.
It seems analogous in many ways to the stages of life we all go through, or even the Heroes Journey. If you want an answer as to why D&D is the most predominate rpg and that mechanics of progress, either levels or skill levels, exist in 90% of rpgs, I think this is the reason.
It's hard wired into our brains isn't it? Why the hell do thousands and thousands of people spend hours of their life playing Farmville?
Noisms mentions some rpgs that are about other things than ambition. I'm unfamiliar with those he mentions. I was thinking about games a little more familiar to me. So what about these:
- Call of Cthulu
- Boot Hill
- Any Superhero Game
I've never played CoC, but from what everyone says about it, it's just a matter of time before your character dies or goes mad. So, what is that about? Is there even such a thing as a CoC campaign or are they by nature necessarily one-offs? I'm not criticizing here, actually curious.
Boot Hill was my second attempt at being a DM, and way back then as a proto-gamer, I was puzzled by what we were supposed to do. Ok, yes, this may have been influenced by my having experienced D&D first. I knew there should be tension, a gun fight, but then what? I wanted the game to give me some kind of framework. I remember very clearly noticing in the rule book how the playtester campaign had set a time limit and a boundary distance: whoever crossed that boundary with the most money by the end of the time limit "won." I remember thinking that very odd at the time (I suppose all law men were run by the DM?) and also thought "Aha, I'm not weird, they had to impose a framework too." So, what's Boot Hill about? And again, does anyone play Western campaigns, or are these all one-offs, situations?
Superheroes! The games I played as a youth (Champions, Marvel, DC) made only the barest nod at character improvement. I mean, Batman did all that before he walked onto stage, right? He doesn't get any better, he's at the top of his form. He may craft individual gadgets to deal with threats but that's different. So, the hero you made, was pretty much always going to be the same hero. So what was that game about? Our games had a lot to do with balancing secret identities/normal lives with hero lives. Also a lot of personal interactions/rivalries within our hero groups.
It strikes me that people really unsatisfied with D&D because there isn't enough roleplaying should really be flocking to hero games. That doesn't seem to be the case though, is it because that genre is traditionally drenched in testosterone, and that there is always the violent confrontations spaced between all the drama?
My hypothesis is that as humans, we tend toward some idea of progress, even artificial, and that games that don't incorporate this (by design or because of genre constraints) end up being one-off games, played as a fun session in-between long stretches of games that do allow for progress.
But I freely admit I may be blinded by my own desires (I crave a sense of progress and accomplishment). In fact, I'm intrigued by the idea of an ongoing game that is about the characters interacting with each other, without worrying about ambition. But I'm having trouble envisioning what the characters would be interacting about, if not goals, and aren't goals the small steps of progress?