Dan talks about the difficulty of balancing specialists (high effectiveness in a narrow field) against generalists (moderate effectiveness in a broad range of fields). This is all very true, but the picture's actually a bit more complicated, I think.
Specialisation and optimisation
I observed in a comment on that post that actually I'm not sure the issue at hand was a specialist vs. generalist one. Nikolai isn't actually not a specialist; he's extremely effective at melée combat, and very poor at ranged combat. He's got a handful of broader skills, because I had to spend the points somewhere and some of the assault marine abilities are rubbish, or because Serious Roleplaying, but you couldn't really describe him as being moderately effective at a wide range of things - the Deathwatch system doesn't really allow for that. He's got about a 30% chance of succeeding at a few specific things that aren't hitting things with swords. He's a little bit less optimised for his specialisation than Iacomo, but that's not really had much impact, because while Nikolai has 99 vocation-related problems, damage ain't one.
What I do think happens is that we have overlapping but incompatible specialisations. All of the characters, as befits Space Marines, are good at killing stuff; the issue is that they prefer very different types of fight. I'll just quote myself to avoid reinventing the wheel:
It occurs to me that part of the reason it's hard to balance may (ironically?) actually be the similarity of our abilities. Deathwatch being a combat-heavy game, it's important to make sure that characters are reasonably balanced in terms of combat influence, which in some cases is DPS, in others seems to include supporting abilities. I suspect this is broadly the case, but I'm not expert. On the whole, Nikolai and Iacomo and Erec are probably more or less equally deadly in a fight across the set of all possible fights.
Unfortunately, balancing numerical effectiveness and strategic effectiveness are a bit different. As you say, given a situation where you have to overcome an obstacle (battle), players and tactical PCs alike are going to look for situations that maximise their effectiveness. In our case that means getting the big guns out at long range; in another party it might mean getting up close and personal ASAP. Because everyone's strength is primarily in combat, everyone's moment to shine is the same occasion, and the maximally effective characters are going to be the most useful.
So being equally good at fighting (and very similar at everything else) on paper, but in very different styles, seems likely to mean the characters whose style is most prevalent will dominate. A similar issue might apply to a ninja in a party of armoured knights - technically equally good at killing people, but stealth attack is never the optimal choice for the party.
All of us get to be awesome and effective in fights, and we are all about equally specialised in doing so. In this case, because the party overall is optimised for killing things at range - and because actually in Deathwatch killing things a long way off is a very good plan - the niche situation of specifically melée combat that Nikolai is optimised for simply doesn't arise. Being specialised, there are few other situations where he is likely to be very effective, and this being the case, the party (as Dan predicts) doesn't tend to spend much time on even those approaches where his 30%s would kick in, because 30% is still not very much.
Contrast this with an alternative specialisation. Let's say Nikolai was a sage. He'd be mediocre in a fight, but when information-gathering or knowledge were important, he'd be the lynchpin of the team. They would excel at different types of activities, but providing each type of activity got some spotlight time, everyone would get to be awesome (providing, of course, you're satisfied with research as your awesomeness).
Or let's say Nikolai was more of a true generalist. There wouldn't really be any tension in the team over approaches, because he'd have no reason to prefer melée combat over ranged. It would also be more likely that he'd either succeed at rolls, or be able to usefully roll at all. Unfortunately (?) Deathwatch really doesn't like generalists, a mentality that I find a little strange because it doesn't especially fit the canon, where Marines are expected to switch between roles regularly and maintain all-round expertise.
Generalising in Deathwatch
Deathwatch uses percentile skills based on attributes: Strength, Fortitude, Intelligence, Perception and so on. Because it's built in such a way as to mimic the statlines in the tabletop wargame, while trying also to adhere somewhat to the somewhat-contradictory canon, this means you have Attributes of 30-40, and therefore skills of about 30-40% if you are trained in a skill at all - which mostly, you aren't.
Additional skills, or skills with a +10 or +20 bonus, can be bought as you gain experience. However, the skills available to you are heavily restricted based on your rank (level), specialty (class) and Chapter (race). Sometimes this makes more sense than others; I'm still puzzled why it's so difficult for Space Marines to learn to blow things up (Demolitions). It's quite hard to get even a +10 on your skill in most cases, and often impossible.
But there's more! Attributes can also be increased with experience, but the cost for doing so varies dramatically by specialty. Assault marines, sticking with out example, pay only 200, 500, 1000 and 1500 points respectively to improve their Weapon Skill (melée), Agility, Strength or Perception, all great for the clash of blades. But they must pay 750, 1500, 2000 and 5000 points for Ballistic Skill, Intelligence or Willpower. That's 9250 points to gain a +20 maximum increase, compared to only 3200 points for their favoured attributes. Most of the specialties have a similar breakdown. At the same time, most skills useful for the combat-oriented work of Space Marines come off Int or Agility.
This means it's not only hard and expensive to acquire in the first place abilities that Fantasy Flight don't consider to be directly relevant to a character's specialty, but also often very expensive to get the related attributes higher to increase your odds of success. You can be mediocre at lots of things, but it's extremely difficult to be competent at a broad range. Meanwhile, your teammates will be really very good at a slightly smaller number of closely-related things.
It's difficult to balance abilities that are significantly different. For example: is it more useful to have a 75% chance to deal 1d20+8 fire damage at 50 feet, or to turn invisible? Is shapechanging into an octopus more or less useful than lulling people to sleep if you can get them to listen to you singing? It's the old familiar Wizards vs. Warriors issue.
A related aspect is when the discrepancy is one of effectiveness rather than type. Games often feature abilities that are very powerful compared to others, and look for ways to balance them. The countermeasures may include cooldowns, taxing, a chance of failure, or risk. The greater the power, the larger the countermeasure.
Deathwatch implements a risk-based system for psychic powers, entirely in keeping with canon. Using magic risks getting hurled around like a rag doll, your face melted, or your soul devoured and your body taken over by a ravening daemon. Because psychic powers are often very powerful, and because the canon dictates it, the risks are considerable.
Unfortunately, I don't think risk is actually a very good way to balance power.
The first issue is that risks are generally distant. In theory, yes, casting a spell might obliterate you from the universe. But it's a really very small chance, and so the vast majority of the time either nothing will happen, or something fairly minor will happen. Psychologically, it doesn't necessarily feel like a threat. When you do, it's more of a gamble than a counterbalance: sure, there's a very small chance that I'll be devoured, but there's a very large chance that I'll be completely awesome. Bear in mind, the Smite power can reliably wipe out a combat squad of Space Marines: used with a push, as it would be against a dangerous opponent, it typically deals 6d10 damage ignoring 6 points of armour to a 6-metre radius, which will fry three or four in one shot.
On the 10% chance of doubles in an Unfettered casting, or always on a Push casting, the Librarian must roll on the Psychic Phenomena table. This does nothing mechanically relevant about half the time. There's about a 30% chance of an effect on the psyker and nearby creatures, a 10% chance of an effect that targets the Librarian only and is capable of actually harming a Space Marine, and a 25% chance of a Perils of the Warp roll.
Perils of the Warp are quite nasty. There's a 1% chance of the psyker just dying, a 10% chance of summoning a daemon prince (very bad), about a 10% chance of the psyker personally taking damage or even permanently gaining Corruption or Insanity points, and quite a large chance of everyone nearby getting pounded with psychic discharge or something. A number of effects effectively stun the psyker for a few rounds.
That being said, you've actually got about a 10% chance overall of something very nasty happening when you Push, and most of those you can survive. Long-term degradation from Perils of the Warp is the main risk.
I think the thing is that when serious risk is relatively unlikely, there's little psychological discouragement from using an ability. A Librarian with Smite has very little reason not to use it all the time, as even on Unfettered (4d10 Pen 4) it will kill many enemies easily with a tiny risk of serious consequences, and is much more effective than their normal weaponry.
The second drawback is that when you incorporate mechanics in a game that are intrinsically risky, there's a very strong inclination from both designers and GMs to mitigate that risk. Thus, the talent Rite of Sanctioning lets you ignore Psychic Phenomena other than Perils, and is a cheap must-have for Librarians - 400 points at 1st level. Fate Points let you reroll a double Unfettered casting that would call for the roll in the first place.
The third drawback is that risks are inclined to conjure up the Paladin Problem - which
I'll discuss in a future post I discuss in this later post.
Back to the party
Coming back to the original discussion... the second balancing issue in our party is that librarians are very powerful, and inadequately balanced. Because of this, and also (in fairness) because K hasn't absorbed a ton of Warhammer 40K canon about how psychic stuff is always bad, she wants to run around constantly frying stuff with her brain; and there's really very little mechanical reason not to. The risks are not particularly big, especially considering we're already in life-or-death struggles; the rewards (easy kills) are considerable.
I suspect part of the idea of Deathwatch is that psykers will balance themselves by avoiding unnecessary use of powers in accordance with the canon, and by restricting the power of their casting. From everything I've read about actual play experience, this is the exception rather than the rule. Librarians cast for preference, librarians push whenever it might be handy. Who can blame them?
This leaves Nikolai overwhelmed on two fronts. The devastator is sufficiently more effective that ranged combat becomes the default tactic, which he's poor at. Meanwhile, his completely unlimited moderate abilities are contrasting with a very powerful set of psychic powers that should be balanced by downsides, but effectively aren't.
Direct and indirect balance
As I said above, different types of abilites can be hard to balance. The librarian has an Augury ability that gives us insight into every single mission; this is really useful from the outset, but it's hard to say how it compares to Smite. Similarly, Nikolai has a selection of general skills that can offer advantages, but in an indirect way. He has lots of tactical skills, stealth and perceptive abilities and some technical knowledge; this sort of thing can allow you to set up advantageous situations that might make a battle easier. At the same time, it's hard to compare them to just being better at fighting, which is fundamentally what you do in Deathwatch.
Moreover, different groups will play in different ways. A group that makes heavy use of Intimidation, Inquiry and other social skills will find those valuable and effective, perhaps avoiding fights entirely or catching enemies on the hop. A group keen on covert ops-style Marining may find that Silent Moves, Concealment and Acrobatics are nearly as important as raw combat ability, and that attention-grabbing powerful attacks are a last resort compared to Stalker boltguns and blades.
All of these different aspects make it very difficult to balance anything, and I have sympathy for the designers attempting it.
Sorry, a bit of a rambling collection of thoughts there. Hope it interests someone.