In this paper we present a new approach for combining several independent trackers into one robust real-time tracker. Unlike previous work that employ multiple tracking objectives used in unison, our tracker manages to determine an optimal sequence of individual trackers given the characteristics present in the video and the desire to achieve maximally efficient tracking. This allows for the selection of fast less-robust trackers when little movement is sensed, while using more robust but computationally intensive trackers in more dynamic scenes. We test this approach on the problem of real-world face tracking. Results show that this approach is a viable method for combining several independent trackers into one robust real-time tracker capable of tracking faces in varied lighting conditions, video resolutions, and with occlusions.