Dwarves and Halflings(First playable appearance: Men & Magic, 1974)
Dwarves have long been a staple of D&D, due to their ubiquity in Tolkien's Middle-Earth stories - major party members in The Hobbit, less so (at least directly) in The Lord of the Rings. These roles have been reversed in the latter, where four of the central characters are hobbits. These two races have often been grouped into similar categories; in original D&D (pre-Supplements), they are both restricted to the fighter class, while AD&D (both 1e and 2e) allow them to be thieves in addition. (1e is unusual in that halflings cannot be clerics, but halfling NPCs are allowed to be druids!)
The reason that I group these two together is as follows: I feel that having both as distinct species is a bit redundant. Short people, sometimes with long beards (including female dwarves, at least according to Gygax), who can't usually be wizards? In my campaign, I've long considered merging the two into a single species, with the subraces having different characteristics; perhaps it's the "Tallfellow" or "Stout" dwarves who are longer in leg and beard.
Elves(First playable appearance: Men & Magic, 1974)
Elves are pretty much essential components of any high-fantasy game. They look and act similarly enough to humans that the two can conceivably adventure together, but are just different enough that a Referee with any creativity can put vast oceans between the two species just under the surface. (This is another reason that the Vulcans on Star Trek tend to pop up as major characters; in fact, one can easily draw parallels between the two). And if these are different species, not just different ethnicities of humans, then it makes sense that they would have some classes that are harder (or flat-out impossible) for them to pursue. An acquaintance pointed out that, logically, they should also be able to do things that humans can't do, and this is why I think multi-classing (with a few limits) is a good way to differentiate demihumans from humans.
Gnomes(First playable appearance: Players Handbook, 1978)
Oh jeez... If dwarves and halflings as separate species seems superfluous to me, gnomes are about as useful as a third thumb. Rarely do I see anyone play them; in fact, I've only encountered two gnome player characters in any fantasy RPG I've run or played (one of which is my own PC, Roywyn Raulnor). Considering Tolkien's influence, it seems fairly obvious that their origin lies in Tom Bombadil, but it doesn't really seem necessary to have yet another species of short height and long beards.
|"I gnow thee gnot, old man."|
(model from Battle for Middle-Earth II)
The main point of interest is that, in AD&D (both editions), gnomes are the only demihuman race that can be illusionists; I concede that my own gnome PC, if ported over to AD&D from her current home of 5e, would be a multi-classed illusionist/thief. But I say just let the combined dwarf/halfling species be illusionists, and free up the gname of "gnome" for fey creatures more resembling those seen on American lawns.
Half-elves(First playable appearance: Supplement I: Greyhawk, 1975)
Half-elves do technically appear in The Lord of the Rings. Elrond is called "Half-elven", although his parentage has little actual impact on his mortality or the way he is viewed by others (at least from what I read - I read all of The Fellowship of the Ring, but couldn't get through more than about a quarter of The Two Towers). He chose to identify with his elven ancestors, and so he's considered an elf.
If a player character wants to be a "half-elf", that's fine by me, but they have to choose whether that means they will identify as an elf or a human; there's no real reason for such a strong level of incomplete dominance that they're considered a separate race. I get the feeling that the desire for half-elves is largely based on min-maxing, as half-elves get more classes to choose from than elves, but still have some special abilities. In that cast, why not just let elves have more classes, and drop the mechanical differences of half-elves?
Half-orcs(First playable appearance: Players Handbook, 1978)
Same here as for half-elves. If a Referee wants to let players be a potentially "monstrous" species, just remove the status of full orcs as mindless, faceless evil minions, and let players be orcs. The games in the Elder Scrolls series, starting with Morrowind, did this with great success; orcs are integrated into society for the most part, and their fierce reputation serves them well as soldiers.
Dragonborn and Tieflings(First playable appearance: Player's Handbook, 2008)
I'm inclined to just say "no". Tieflings have a bit more history, appearing initially in the Planescape setting for AD&D 2e, but dragonborn have no excuse aside from Wizards of the Coast trying to cash in on the humanoid races popular in World of Warcraft. If someone wants to play a dragon-like character, there were already half-dragons for over ten years by the time 4e came out! Also, as Preston Selby pointed out here:
"I just think there's a sort of breakdown in the game when a player can say their character is a half-dragon with a horny lizard-head and a breath weapon, and there is an expectation that the character can walk into a town and an inn with the humans and the halflings and everyone will act like this is totally normal. At that point, the game has seriously damaged its potential for weirdness and wonderment."At that point, discrimination by non-fire-breathing humans and dwarves isn't necessarily based on irrational, prejudicial fears (although there's probably an element of that); it's a very rational fear that the dragonborn might sneeze too hard and burn down your house!
I realize that this gets into the same thorny area as X-Men (as much of a prick as he was, Senator Kelly was right when he referred to powerful mutant teenagers as "weapons in our schools"). This is why fantasy and sci-fi can only use metaphors for racial and religious discrimination up to a point.
Ack, back on topic: I don't think dragonborn should be a "standard" species in the kind of games I like to run. As always, these are just my opinion, not some kind of holy pronouncement.