Biologically Inspired Camera Positioner
| Mechanism |
The mechanical design of the robotic vision system is inspired by the human ocular system. It has a single-degree-of-freedom (DOF) positioned by an antagonistic pair of cellular actuators. Each cellular actuator consists of 16 PZT actuators with the multi-layer elastic strain-amplification mechanisms that allow for a fast response and wide range of motion. When the rod is positioned by the antagonistic pair of the cellular actuators, the moving platen is rotated with respect to the pivot axis. The range of the angular displacement is approximately ±13◦. As a result, the camera can be positioned within 60 ms with the maximum velocity of 350 degrees per second that is comparable to the speed of human saccades. To avoid the hysteresis in PZT, each actuator is operated in an ON–OFF manner. Simply choosing how many PZT actuators to activate will achieve a desired angular displacement. This approach is similar to the physiological bang–bang type of inputs given in saccadic eye movements. In this way, collections of cellular actuators share principles with muscles in which their action is a sum of the individual motor units that are recruited.
| Discrete Switching Vibration Suppression |
The camera positioning mechanism exhibits a single dominant frequency which was determined by simple experiments. Therefore, the system can be represented as a linear second-order system. A saccade-like motion is an open-loop controlled point-to-point motion with 50ms settling time at the maximum angular velocity of 250 deg/sec. In order for camera positioning mechanism to generate the saccade-like motion, quantized input commands must be given at appropriate times to the piezoelectric cellular actuators with successful vibration suppression. This can be achieved by understanding the mechanical properties of the system such as the natural frequency and the damping ratio. By applying a phase-vector analysis of mechanism control, discrete switching commands can be given to the system. The results show that the vibration has been suppressed and the settling time of each motion is within 60ms. If a step command is given to the system, however, the response shows considerable oscillation due to the lightly damped system. The maximum peak velocity of the camera orientation system given the discrete switching commands is approximately 350 deg/sec. Therefore, the speed of response of the camera positioning mechanism is comparable to that of the human ocular motion.
[youtube_sc url=https://www.youtube.com/watch?v=jStjK4NTrRs&feature=youtu.be]
| Dynamics-based Image Processing |
Saccades are known to be the fast and rapid eye movements enabling quick scanning the field of view and high-resolution fovea at the region of interest. Despite of saccadic eye motion, we are almost unaware of potentially blurry visual input. The neural system blocks images with substantial motion blur in the course of the eye motion, termed saccadic masking. Saccadic masking plays a key role when the motion of the eye is not negligible in visual perception. The equivalent frame rate of the human eye is reported to be at most 24 fps (varies depending on situation) or 42ms/frame. Research shows that a typical saccadic (point-to-point) eye movement completes within 20-80ms, comparable to the rate of visual perception. Eye movements are primarily self-induced, i.e., generated by the extraocular muscles receiving commands from the oculomotor system. We hypothesize that there is a neural mechanism that makes a prediction of such self-induced eye movements based on the dynamic model of the eye and enables effective image de-blurring and panoramic image stitching, or in short, dynamics-based visual processing. We consider this possible mechanism is the key enabler of real-time panoramic image stitching with simultaneous motion de-blurring in a dynamic vision system, allowing for generic image sensors with a standard frame rate, and requiring no motion sensors and significantly less computational load.
We extended the dynamics-based image de-blurring method to image stitching by visual-motor coordination. The robotic vision system was capable of quick scanning the environment to generate a panoramic image, since saccade-type motion was achieved. Conventional image-based stitching methods have been effective. However, they are not robust to degraded blurry images due to incorrect extraction and matching of features.
| Velocity Control (Smooth-Pursuit) for Tracking an Object |
The actuator system exhibited compliance and quantization and thus extensions in discrete switching controllers were necessary. A feedback control with pulse-width-modulation (PWM) is a modulation (PWM) is a classical and simple method to operate an electromagnetic system with discrete switching. However, it causes high-frequency switching in individual PZT actuators that would damage the structures. In addition, this approach has little in common with biological muscle actuation. For this reason, another extension of the discrete switching vibration suppression (DSVS) methods are used to compute when to activate and de-activate actuator units to generate an arbitrary velocity profile. Muscle-inspired impulsive actuation was developed to generate arbitrary velocity profiles for tracking an object of interest. Unlike other visual tracking methods for feeding video, the proposed method is designed to match the velocity of the object of interest only during the exposure window. Therefore, the residual vibrations in the cellular actuator are suppressed while reaching to the next discrete position. This approach allows to decrease numbers of switching further.
| Members |
Sean Kilbride
| Publications |
Journal Papers
Michael Kim and Jun Ueda, “Real-time panoramic image generation and motion deblurring by using dynamics-based robotic vision,” IEEE /ASME Transactions on Mechatronics, Volume 21, Issue 3, pp. 1376 – 1387, 2015
Michael Kim and Jun Ueda, “Dynamics-based motion de-blurring for a PZT-driven, compliant camera orientation mechanism,” The International Journal of Robotics Research, Vol. 34(4-5) 653–673, 2015
Joshua Schultz and Jun Ueda, “Nested piezoelectric cellular actuators for a biologically inspired camera positioning mechanism,” IEEE Transactions of Robotics, , Vol. 29, No. 5, pp. 1125-1138, 2013
Joshua Schultz and Jun Ueda, “Two-Port Network Models for Compliant Rhomboidal Strain Amplifiers,” IEEE Transactions on Robotics, Vol.29, No.1, pp.42-54, Feb. 2013
Joshua Schultz and Jun Ueda, “Experimental Verification of Discrete Switching Vibration Suppression,” IEEE/ASME Transactions on Mechatronics, Volume 17, Issue 2, pp. 298 – 308, April 2012
Conference Papers
Michael Kim and Jun Ueda, Discrete switching commands for tracking and vibration suppression using a quantized, compliant camera orientation system, 2016 IEEE International Conference on Robotics and Automation (ICRA2016), Stockholm, Sweden, May 2016
Michael D. Kim and Jun Ueda, “Real-Time Image de-blurring and Image Processing for a Robotic Vision System,” 2015 IEEE International Conference on Robotics and Automation (ICRA2015), Seattle, May 2015
Michael Kim and Jun Ueda, “Dynamics-based Motion Deblurring for a Biologically-inspired Camera Positioning Mechanism,” Proceedings of 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS2013),Tokyo, Japan, November 2013
Sean A. Kilbride, Michael D. Kim, and Jun Ueda, “Real Time Image De-blurring and Image Stitching for Muscle Inspired Camera Orientation System,” Proceedings of the 2014 IEEE Workshop on Advanced Robotics and its Social Impacts (ARSO 2014), Evanston, USA, September 2014
Joshua Schultz and Jun Ueda, “A Camera Positioner Driven by Muscle-Like Actuation,” The 2012 IEEE / RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob 2012), 2012
Joshua A. Schultz and Jun Ueda, “Analysis of antagonist stiffness for nested compliant mechanisms in agonist-antagonist arrangements,” the 2011 ASME Dynamic Systems and Control Conference (DSCC’11)
Joshua Schultz and Jun Ueda, “Intersample Discretization of Control Inputs for Flexible Systems with Quantized Cellular Actuation,” the 2010 ASME Dynamic Systems and Control Conference (DSCC’10), Boston, MA, 2010
Joshua Schultz and Jun Ueda, “Discrete Switching Vibration Suppression for Flexible Systems With Redundant Actuation, ” 2009 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM 2009), Singapore, July 2009
Patents
Camera positioning mechanism using an antagonistic pair of compliant contractile actuators
US 8662764 B2
Press Releases
- Robot Vision: Muscle-Like Action Allows Camera to Mimic Human Eye Movement, GT News, July 2012 (also introduced by a number of media outlets)
- Camera Uses Muscle Like Action to Mimic Human Eye Movement, Research Horizons, Georgia Tech Research News, Fall 2012-Winter 2013
| Funding |
Funding for this research is provided by the National Science Foundation (NSF) grant from the Dynamical Systems (DS) program in the Division of Civil, Mechanical, and Manufacturing Innovation (CMMI) for the research on non-uniformly quantized actuation in biological motion generation (#1300019). The research will result in an in-depth understanding of how modularity and variability in the neuromuscular system play key roles in coordinating multiple muscles. Specifically, this project will establish a physiologically inspired framework for recruiting compliant modular actuators for biological movement generation.
