Last month, on Magical Words, Faith Hunter began posting a series about the requirements of a successful modern antagonist in several different genres. She calls the series The Great Satan, and has so far posted three parts. I looked forward to the continuance after ConCarolinas is over.
The primary thrust of her articles is that, in most genres, antagonists need character development and motivation as much as the protagonist. They are, after all, equal and opposite.
This aligns very well with what I have been trying to be more conscious of: making sure all of my major characters have a reason to be there. This idea also echoes my acting training very well. When on stage (or camera), every single movement or action or expression means something. Thus, you cannot do anything on stage (or camera) without a reason for doing it. Too many beginning actors make this mistake, moving across the stage because "they felt like it" or, worse, because "I need to be there when Erick enters the scene 5 minutes from now." Your character doesn't know Erick will be entering the scene in 5 minutes, or that he needs to be standing there to catch Erick when he falls in, drunk! So, the actor needs to be there, and the actor needs to find the character a reason to be there, too.
So now we see that this is a mistake a writer can make as well. Why does the dragon destroy the land? Why does the queen kill any pretty girl in the kingdom? Why does the magic tower of smoke drag people down the hole? It might be a dragon, a government employee, or a magic tower of smoke, but it has to have a reason for doing whatever it is that it's doing.
Faith Hunter asks four questions as she addresses the common antagonist in each genre:
1. What makes them work?
2. How do we keep them from becoming formulaic? (The pseudo-Satan.)
3.What mistakes do we writers make that allow them to become formulaic? (Just another way of looking at number two above, with a different perspective.)
4. And how do we as readers contribute to the success or failure of the BBU? (Culture and the reader.)
Based on the first posts, and my own experience, I would say that generally, the answer to #4 is "expectation of the genre" and "suspension of disbelief." The reader knows if they pick up a mystery, that there's going to be a murder and the guy's going to try to get away with it. If they pick up a fantasy, there's going to be magic involved, especially from the "Big Bad Ugly."
The other three questions, however, are what we writers need to pay attention to in this realm. Much of the answer falls into the idea of developing the antagonist the way one would (should) develop their protagonist: make them believable, give them proper motivation, paint them with both weaknesses and strengths, etc. In other words, make your "villains" into "characters" instead. Give them a reason to be in your book other than what you need from them.