I've heard that argument often. Unfortunately, it has a hidden premise: Choosing based on who is best for the job, requires some shared values about what the job is.

Let's oversimplify the world by assuming that there is only one decision to be made by a particular office holder. If you had a strong stand on that issue, then wouldn't you prefer an incompetent who shared that belief and was likely to move things infinitesimally in your preferred direction to a brilliant and skilled leader likely to take the country far in the opposite direction.

Of course the real world contains a mixture of issues where there is sharp disagreement and those where overwhelming majorities in favor of one view. The more competent person will be better on the consensus issues, but more often than not, I find the partisan issues to be more important, so it hasn't been unusual for me to find myself voting for the lesser person because I agreed with him more.