Don't do stupid evil

You might remember that scene from the book. You might remember it from the movie, too. That single scene when a villain has succeeded against the hero and remains there gloating, being able to finally kill the hero (or his pet turtle)... yet he didn't. And the person watching the movie is already like "JUST KILL HIM ALREADY!"

Even when the villain is a remorseless boss of mafia or a dictator who has killed hundreds before.

A variant of this would be "Ah-ha! I have you in my mercy! I know! A honorable duel to the death!". Or giving a chance for a hero to escape. In other words, a situation when a villain acts irrationally only to give the hero a chance "because of a story".

Sucks from the perspective of a viewer who expects consistent, rational agents.

The problem

Why is it a problem? Because this is a situation a lot of GMs are put into. Either GM wants to show the players "the final boss" (and not risk the boss' accidental demise) or the boss has an upper hand on the players and GM does not want (or should not) kill the players.

So what the GM is going to do - usually - is to invent some kind of insane reason why the villain doesn't want to kill the players (or why the players should not attempt to kill a villain). Unfortunately, this type of reasoning usually makes the whole plot (or story, or encounter) look very stupid instead of interesting and nerve-wracking.

And GMs usually really, really dislike pointless and accidental deaths of key characters - be it player characters or NPCs.

Sucks from the perspective of a GM who doesn't like killing player characters and likes to have some tension.

The escalation

Let's assume we have a GM who does not like to kill players' characters and wants to create strong, cunning, reasonable villains.
Let's assume we have trigger-happy players who tend to maximize the efficiency of their actions.
If we assume the GM wants to show the villain to a player... the situation is unresolvable.

Intelligent players will try to assassinate a villain like that, either using their own abilities or hiring the best assassin (I would use poison, personally). An intelligent villain should try to lure the players into a building full of explosives and simply blow it up leaving no chance to escape (or use poison).

In other words, in confrontation of two cunning and powerful forces who are in direct collision, one force should slay the other. No sweet-talking, no "let's give other side a chance". Just - cold and efficient execution. Like player characters cutting through the goblins - a villain is just an obstacle, nothing more.

So rationally this situation is unresolvable under the given constraints. Can it be resolved if we changed the constraints?

Changing the reality

Having this situation - how to make two cunning and powerful forces NOT kill each other? Simply put, killing the other side must be the losing strategy, or at least, not a winning strategy.

For example - "what if the villain and the player characters could not perma-die?"

Actually, this is what is happening in the ficion - the key characters do not die (until the villain does, of course). But usually the characters do not die because of the incompetence of one side. Let us make it more interesting then.

"What if the key characters' death was harmful for the opposite side?"

The goalpost has moved - right now the villain wants to absolutely prevent the player characters' death and the player characters want to prevent the villain's death. An example: "as long as villain lives, player characters are immortal" or "the villain has a dead man's hand: if he dies, nuclear bomb explodes somewhere nearby".

What if...

We are still talking about fantasy and sci-fi worlds, right? Let's make it more fun.

"What if noone in the world could ever die forever? What if someone is killed they return to life in the cloning chambers?"

Now it is interesting. Although GM can kill player's characters, they will not die forever; they will "respawn" in a safe spot. The characters can kill the main villain (and infinite amount of mooks); they all will return anyway. This creates a world of reckless actions, one, where you want to hit enemy cloning chambers (actually, all of them) and THEN kill the original villain. A different world, one where death is used as a light detriment; death penalty would be something meaningless while incarceration and maiming of the remaining person would be a very harsh penalty.

"What if people can die, but after they are killed they become immortal revenants killing all living for 3-4 hours?"

This reality is the one where no concept of "armies" or "cities" has ever been created. Imagine an army: 1000 people on one side, on the other side - a lone sniper. Sniper kills a single opponent, he returns as a revenant and kills another. Those two kill two others, then four others die...
Actually, a small bar brawl could annihilate a city.

"What if revenants return only if people die having hatred in them?"

Now the sniper is largely neutralized; though every army has a chaplain (or several chaplains) who have an extremely important role of keeping people's compassion up and eradicating anything which could lead to hate...

The conclusion

Note that whatever the answer to the "What if" question is, every single answer creates an interesting, distinct world.
What is important - the game world needs to answer your players' needs. All those previous situations come from the problem "GM does not want a perma-death for key characters", yet every single of them creates a different world with different strategies for cunning and rational characters to combat their opponents.

Whatever you want your system to do, whatever you want your reality to be, at one point every GM will have to face the fact that the story might not be aligned with the logical choices the intelligent characters should make. And at that point the world will promote a rational solution - one which corresponds to the reality of this world.
Don't make your characters stupid. Change the reality, so everyone is aware why the character is intelligent and powerful.

In the world where people cannot die a rational villain will not hit the players directly - he will hit their plans, goals and resources.

Or will play hide-and-seek with their cloning chambers. I can imagine an exciting campaign where both sides try to destroy cloning chambers of the other side, and at one point the last remaining clone tries to hide and not die until a new cloning chamber is acquired... and imagine the satisfaction on the players' faces when they FINALLY take care of their supposedly invincible nemesis.

In other words, take what bothers you in your current session / campaign. Name it. Ask "what if it was impossible". Try to create a reasonable explanation which fits your game world. Change your reality. Noone will ever sneer at the major villains in your campaign anymore.

It is that simple.