*NEWS*: since June 2016 vision-ary project joined ARGO Vision, an innovative firm that excels in visual recognition. For info about the cascades, please send a message via contact form here.

For any other info about Computer Vision and Artificial Intelligence please contact ARGO Vision.

In this note it is discussed how real-time face tracking in video can be achieved by relying on a Bayesian approach realized in a multi-threaded architecture. The system is based on a probabilistic interpretation of the output provided by a cascade of AdaBoost classifiers. Results show that such integrated approach is appealing with respect either to robustness (please note how the wrong hypothesis are going to be deleted and how the almost-corrected ones are “tuned”) and computational efficiency (in 2009, on a dual-core PC the face tracking is ~100 fps)

Face tracking can be performed either according to a frame based approach (e.g., [1]) or according to a detection and tracking approach, where faces are detected in the first frame and tracked through the video sequence (for a review, refer to [2]). Clearly, in the first case, temporal information is not exploited, and the intrinsic independence among successive detections makes it difficult to reconstruct the track of each subject. In the second case a loss of information may occur (e.g., new faces entering the scene) and, in general, the output of face detection is used only at initialization, while tracking relies upon low level features (color histograms [3], contours [4], etc.) which are very sensitive to the conditions of acquisition. To overcome these drawbacks, a tighter coupling between face detection and tracking has been proposed [2]. Such an approach can be given a simple and elegant form in the Bayesian framework.

Each face is characterized at frame t of the video stream by a state vector xt, e.g., a face bounding box. The tracking goal is to estimate the correct state xt given all the measurements Zt = {z1, … , zt} up to that moment, or equivalently to construct the posterior probability density function (pdf) p(xt|Zt). The theoretically optimal solution is provided by recursive Bayesian filtering that in the prediction step uses the dynamic equation and the already computed pdf of the state at time t−1, p(xt−1|Zt−1), to derive the prior pdf of the current state, p(xt|Zt−1); then, in the update step, it employs the face likelihood function p(zt|xt) of the current measurement to compute the posterior pdf p(xt|zt). Thus, the key issue of a tight coupling approach is to provide a face detection algorithm suitable to calculate the face likelihood function p(zt|xt) and, if a Particle Filtering (PF,[4]) implementation is adopted, to generate proper particle weighting. For instance, Verma et al. [2] adopt the Schneiderman and Kanade detector [5], which provides a suitable probabilistic output, but unfortunately is inadequate for real-time implementation.

A more appealing solution, which we propose here, could be the real-time face detection scheme proposed by Viola and Jones (VJ,[6]) – basically a cascade of AdaBoost classifiers – which is arguably the most commonly employed detection method [7]. Note that the combination of AdaBoost and PF has been proposed in [3], but detection results were heuristically combined in the proposal function, rather than being exploited to model a likelihood function in a principled way. In [8], PF is integrated with an AdaBoost monolithic classifier via the probabilistic interpretation given by Friedman et al. in [9]. Unfortunately tight coupling is spoiled by the introduction of a mean shift iteration, based on color features, to support the prediction of new hypotheses in adjacent regions.

Differently, we exploit the cascaded classifiers described in [10]. In this case, the probabilistic interpretation valid for monolithic classifiers does not apply directly; thus we work out a probabilistic interpretation of the VJ algorithm suitable for our purposes. We discusses the proposed PF implementation by defining the state parametrization and the adopted dynamical model. A PF may perform poorly when the posterior is multi-modal as the result of ambiguities or multiple targets, and issues such as appearance and disappearance of faces should be handled in a robust way. Interestingly enough, the latter issue, which is considered critical in PF based tracking, is very easily solved in human vision where multiple object tracking “runs” in parallel with background motion alerting processes capable of triggering, through pop-out effects, the tracking of new objects entering the scene and the loss of attention for disappearing objects. These problems have been tackled by incorporating such concurrency of processes in the system architecture.

References