Autonomous Target Following

Overview

Our motivation is to build autonomous robots that can follow people and recognize their activities. Such capabilities are important in applications such as home care for the elderly, intelligent environments, and iteractive media. In particular, the goal of this project is to develop reliable and efficient motion strategies for an autonomous robot to follow a target and keep it within sensing range. In this research, we investigate the use of greedy strategies using local information to acheive the goal.

A typical target following scenario in an everyday environment, e.g., the school cafeteria.

People walking around create a highly cluttered and dynamic environment in which the robot has to follow the target.Quicktime video (3.4 MB)

Greedy Strategies

Greedy tracking is useful, when very little is known about the environment and the target. Without an a-priori map of the environment, the tracker must acquire local information on the target and the environment using its sensors. In this work, we assume sensors based on a line-of-sight visibility model, e.g. laser range finders, cameras, etc. In particular, we use a SICK LMS200 laser range finder mounted on a robot with differential drive as the tracker in our experiments. The target is a person walking around a crowded environment. The target’s intentions are also unknown in advance, in particular we focus on non-adversarial targets.

Tracker : A SICK Lms-200 laser mounted on a Pioneer-3DX base

The robot’s local information.

Tracking Strategy

The objective of tracking is to keep the target within the tracker’s visibility region at all times. The tracker’s visibility is occluded by the presence of environmental obstacles and the sensing range of the tracker’s visibility sensor. The target can escape the tracker’s view through the gaps in the visibility region, also known as gap edges.
A line-of-sight visibility region has gaps due occlusions and the sensor limits, Range and Field-of-View (FoV). The danger zone around the gap-edge is the region where the target has higher relative vantage than the tracker. The robot balances the short term goal of swinging and long term goal of moving towards the occlusion edge while following the target along a corner. Quicktime video (2.4 MB)

To select actions effectively, the robot must balance between the short-term goal of preventing the immediate loss of the target and the long-term goal of keeping it visible for the maximum duration possible. The tracker does so by intelligently maneuvering the gap-edges away from the target. The ability of the tracker’s maneuvering power, and hence its ability to track the target, depends on its relative position and velocity w.r.t the target towards the gap-edge. This generates a notion of relative-vantage w.r.t the target. We define a region where the target’s ability to escape is greater than the tracker’s ability to eliminate the gap-edge as the danger zone. And so, the target has a higher vantage inside the dangerzone and the tracker outside. Taking this vantage as the objective function, the tracker can optimize its velocity to improve its relative vantage w.r.t the target.

Local Prediction

We predict the target’s motion locally in order to choose actions to give higher weightage to more probable escape points. Although many prediction methods are possible, we use a simple one in which the target’s velocity in the next step is estimated by storing a limited history and extrapolating, and the uncertainty of the target’s velocity estimation is modeled as a probability distribution.
Without prediction the attention of the tracker is spread over all the escape edges. A simple prediction lets the tracker focus on the most probable escape point. The simple probability measure is applied and it generates reliable relative probability of escape of the target through the escape edges Quicktime video (1.6 MB)

2-D Target Tracking

Experiments show that such a tracking strategy performs better than those that just servo onto the target or those that just take the shortest distance to escape of the target into account. Below, the red robot is the target being followed by the green tracker. We see that the vantage tracker is able to track the longest.

Visual Servo Shortest distance based tracker Vantage tracker
The visual servo algorithm does not take into account the changing environment and allows the target to be occluded by the box. Quicktime video (1.7 MB) The vantage tracker explicitly models the visibility relationship of the target, itself and the occlusion due to the box at each instant and successfully follows the target. Quicktime video (3.0 MB)

Stealth Tracking

An advantage of our fomulation for the tracking problem in terms of objective function is the ease with which additional considerations can be accomodated. For example, stealth tracking requires the robot to follow a target without letting itself exposed. In this aspect stealth is actually the inverse of the tracking problem. Our algorithm applies a stealth constraint to the tracking objective function. The stealth constraint requires the robot to stay close to the boundaries of the target’s visibility while keeping the target away from the robot’s visibility boundaries. Due to local sensing, the target’s visibility can only be computed within the robot’s own visibility. We impose a lookout region (L) around the target’s visiblity boundaries in which to constrain the optimization problem.
Lookout regions due to the target’s visibility. Constrained optimization causes the robot to move in a stealthy fashion. Quicktime video (1.7 MB)

Hardware Limitations

The limitations of the hardware also affects the tracking problem. E.g, to cater for the limited field of view (FoV) of a sensing mechanism, the robot must in addition make sure that the target is away from the FoV limits. The tracking strategy considers these limits as pseudo escape edges and tries manipulate these away from the target in a similar fashion to the occlusion edges. To validate our algorithm, we bring the robot to follow a person in the school cafeteria.
The target (red pioneer) is followed in an indoor environment by the robot (green pioneer). The right window shows the visibility and decision making process for the robot. Blue lines denote the escape edges and the green line shows the current decision velocity. Note that when the target marked by T, goes too near the edge, the robot swings to keep it in view. The most critical edge is marked red. Quicktime video(7.4 MB) The canteen environment is inherently complex with the table and chair legs having a similar signature as the target person’s legs. People walking by create visual occlusions and mobility obstructions. Quicktime video (2.2 MB)

Robustness of the vantage tracker

Due to the fast online nature of the vantage tracker, the errors due to uncertainty are bounded, and it is able to recover from temporary occlusions. Such a tracker is also robust to uneven terrains.
Temporary occlusions: A challenging aspect of following the target in a crowd is when someone walks in between the robot and the target. In this video the robot is following the target in green t-shirt when if faces an temporary occlusion by a lady walking across (purple). These temporary occlusions in such a dynamic environment is handled well by the robot. Quicktime video (1.8 MB) Uneven terrain: The robot follows the person successfully in uneven terrains. Notice the high fluctuations in the sensor readings in the bottom right window. The fast recomputation and online nature of the robot helps it in maintaining the target as the robot takes a bumpy ride. Quicktime video (1.5 MB)

Tracking in 3D Environments

A related approach has been developed for tracking in 3-D. Although the basic principles are similar, the 3-D problem is much more difficult. Moreover, there are situations where pure local greedy strategies may not work well.

The consideration of relative vantage in 3-D leads us to vantage zones as shown. The robot tries to manipulate these vantage zones in 3-D leading to tracking motions.
Tracking scenario in 3-D. A single occlusion plane (in blue) being generated due to the obstacle (pink wall). The blue sphere denotes the robot position and the red cube the target. Vantage zone in 3-D. R is the robot, O the occlusion plane, Oe is the lateral edge and g, the occlusion edge. The grey prism represents the obstacle.

Tracking in an urban scenario. A tracking helicopter follows the target in an urban environment using a 3-d scanning laser data points. 3-D Visibility polygon is extracted from the data point cloud. The motion is then generated to optimize risk based on the vantage zones. Quicktime video (3.3 MB)

References

  • T. Bandyopadhyay, D. Hsu, and Ang Jr., M.H.. Motion strategies for people tracking in cluttered dynamic environments. In Proc. Int. Symp. on Experimental Robotics, 2008. To appear.
    PDF
  • T. Bandyopadhyay, Ang Jr., M.H., and D. Hsu. Motion planning for 3-D target tracking among obstacles. In Proc. Int. Symp. on Robotics Research, 2007.
    PDF
  • T. Bandyopadhyay, Y.P. Li, Ang Jr., M.H., and D. Hsu. A greedy strategy for tracking a locally predictable target among obstacles. In Proc. IEEE Int. Conf. on Robotics & Automation, pp. 2342–2347, 2006.
    PDF
  • T. Bandyopadhyay, Y.P. Li, Ang Jr., M.H., and D. Hsu. Stealth tracking of an unpredictable target among obstacles. In M. Erdmann and others, editors, Algorithmic Foundations of Robotics VI—Proc. Workshop on the Algorithmic Foundations of Robotics (WAFR), pp. 43–58, Springer, 2004.
    PDF

People