Why Doesn't the Valeyard Work?

(Cross-posted from Fraggmented.)

In 1986, during Season Twenty-Three's 'Trial of a Time Lord', Bob Holmes (with contributions from others including Philip Martin, Eric Saward and Pip and Jane Baker) introduced a character that instantly gripped the imagination of pretty much the entire fanbase at the time...the Valeyard. He began the story as the prosecutor in the titular trial of the Doctor, but by the end he was revealed as something far more shocking--a future incarnation of the Doctor, a distillation of all his worst impulses into living form. He instantly became a major, core element of the mythos of the series...

Well, no, actually he didn't. In fact, apart from a few audio stories (I think that Beep the Meep may actually have appeared more often than the Valeyard) and a mention in the Season Seven finale, the Valeyard has been rather conspicuously absent for a character who would seem to have so much storytelling potential. In point of fact, for the longest time he was not only absent but forbidden: The Virgin submission guidelines made it clear that any pitch featuring the Valeyard, explicitly stating that they felt he had no storytelling potential and was a crutch used by bad writers in order to make their stories seem more significant.

Is that true? Certainly, you could argue pretty persuasively that any story that features the Valeyard could be done just as easily with the Master; he's already the Doctor's "dark mirror", so in a lot of ways the part is already taken. (It's probably significant that the one major Valeyard story featured the Master helping the Doctor against the Valeyard.) But surely there has to be something that can be done specifically with the Valeyard that can't be done with a generic "evil Time Lord scientist", right? There has to be something particular and special about the idea of the Doctor's potential corrupted and debased into cruelty and sadism?

But the Valeyard we see on screen has nothing to him beyond cruelty and sadism. He's evil. Full stop. The Doctor's "dark mirror" is a murderous sociopath who does evil things for evil's sake, or at least that's how he's played in 'Trial'. He's a sneering, preening, gloating villain who wants to cause chaos for its own sake. If he's the Doctor's dark mirror, then the Doctor must be a humble, self-effacing sort who's interested in preserving order and..

Ah. Yes. There it is.

The Doctor has never been an unambiguously, uncomplicatedly "good" individual. He's a mercurial creature of chaos in his own right, toppling governments and dashing off into the night without ever caring what results he leaves behind. He's at times callous, at other times startlingly sympathetic over trivial details. He's refused to kill his enemies because he believes deeply in compassion...and he's steered whole fleets of alien conquerors into the sun with a casual "good riddance". He's burned whole planets, and sacrificed his life to save a single man. He is perhaps the most strikingly complex protagonist in television history...and yet his "dark mirror" is just a typical Man in the Black Hat who comes up with complicated-yet-rubbish schemes. (Am I talking about the Valeyard or the Master? Yes.)

For the Valeyard to work, he'd have to be far more like the Doctor than he is. He'd have to be a capricious monster, one just as willing to spare an entire world from his depredations simply because he liked the color of the sky as he was to crush a sparrow underfoot for singing out of tune. He'd have to be an agent of order as well as chaos, perfectly willing to spend decades on a trivial task because it was worth doing right and then dashing a whole civilization to dust with a few whispered words. In short, a dark and twisted incarnation of the Doctor would be very difficult to distinguish from...the Doctor. The difference between the Doctor's best self and his worst impulses is a matter of degree and emphasis, as he himself has admitted on occasion. ("Good men don't need rules. Today is not the day to find out why I have so many.") Ultimately, the reason the Valeyard is so underused is because he's superfluous to requirements. The Doctor has all the darkness he needs without having to outsource it.