Much has been written on this subject and for good reason, because the church needs not just good men it needs good men who will lead people to Christ. Leading people to Christ inherently requires leadership. The Christian life is not one of separation and study alone. It is one of interaction and guidance. A pastor is not a monk he is a follower who leads other followers. He is flawed for sure but he must not let his flaws disqualify him from leading but must lead in such a way that he points beyond himself to Christ.
How many people follow you doesn’t make you a leader THAT someone is following you does. If you are a pastor you are a leader. You have a responsibility to lead well and to lead people to a closer walk with Christ.