Some aggressive cats direct their anger almost exclusively towards people.
This type of aggression in cats usually occurs if the cat as a young kitten either wasn't handled and petted enough, or was mistreated.
He's mistrustful and wary of humans as a result, so tends to lash out whenever he feels threatened by one.
The key to resolving this aggressive cat behavior is to get him to trust you. He needs to get used to being touched, and to realize that nothing bad is going to happen to him when you approach him.
If he won't let you touch him at all, wait until he's eating and then slowly approach him and sit down as near as you can without putting him off his dinner.
Over the following days and weeks, gradually move closer. Start to do the same thing when he's sitting relaxing. Slowly, he should learn that you're not a threat to him.
When you can sit next to him, and he's relaxed and content, start to get him used to being touched. Stroke his head first, then his back.
Talk to him and do everything slowly. Hand feed him a treat. If he looks like he's becoming agitated, move away.
Try to get him to play with you. Attach a toy he likes to a long piece of string, and slowly pull it towards you. Gradually, he should become more relaxed about being closer to you.
This may be a slow process for a cat who's been unlucky enough to have a very bad start in life...
However, if you're able to persevere and either cure your kitty or at least partially reduce his fear, you'll get a great sense of satisfaction knowing that you've given him a much better quality of life.