Correct. And fans were skeptical of both hires because of it. They both worked out and were great coaches. Both were also heavily considered for the other jobs too - Dooley almost left for Auburn in '81 (they made him a huge $ offer); Dye almost left to replace Dooley in '89.
I've always thought the "he played/coached at our rival" stuff to be overblown by fans.