The pain of my wife with another man was more than the pain of betrayal, is this normal?

These two issues do indeed overlap and are normal considering what has happened. Perhaps you can talk to a trusted counselor or therapist to help you through all of this. Then help you move on. It takes time, but it will happen. I wish you luck.