In this project, we aim to construct a camera network of RGB and RGB-D sensors (Kinect) that can be efficiently streamed across the IP network. We will develop and implement robust algorithms (based on sparse and low-rank pose/appearance representations) that enable persistent tracking across the camera network. Moreover, by accumulating and transferring the RGB-D model of each moving individual from one camera view to another, 3D pose can be persistently estimated. Our proposed method will be applied to multi-human markerless augmented reality in the NexCave of the VCC center.