View Chapter

Chapter 40 — Mobility and Manipulation

Oliver Brock, Jaeheung Park and Marc Toussaint

Mobile manipulation requires the integration of methodologies from all aspects of robotics. Instead of tackling each aspect in isolation,mobilemanipulation research exploits their interdependence to solve challenging problems. As a result, novel views of long-standing problems emerge. In this chapter, we present these emerging views in the areas of grasping, control, motion generation, learning, and perception. All of these areas must address the shared challenges of high-dimensionality, uncertainty, and task variability. The section on grasping and manipulation describes a trend towards actively leveraging contact and physical and dynamic interactions between hand, object, and environment. Research in control addresses the challenges of appropriately coupling mobility and manipulation. The field of motion generation increasingly blurs the boundaries between control and planning, leading to task-consistent motion in high-dimensional configuration spaces, even in dynamic and partially unknown environments. A key challenge of learning formobilemanipulation consists of identifying the appropriate priors, and we survey recent learning approaches to perception, grasping, motion, and manipulation. Finally, a discussion of promising methods in perception shows how concepts and methods from navigation and active perception are applied.

Catching objects in flight

Author  Seungsu Kim, Ashwini Shukla, Aude Billard

Video ID : 653

We target the difficult problem of catching in-flight objects with uneven shapes. This requires the solution of three complex problems: predicting accurately the trajectory of fast-moving objects, predicting the feasible catching configuration, and planning the arm motion, all within milliseconds. We follow a programming-by-demonstration approach in order to learn models of the object and the arm dynamics from throwing examples. We propose a new methodology for finding a feasible catching configuration in a probabilistic manner. We leverage the strength of dynamical systems for encoding motion from several demonstrations. This enables fast and online adaptation of the arm motion in the presence of sensor uncertainty. We validate the approach in simulation with the iCub humanoid robot and in real-world experiment with the KUKA LWR 4+ (7-DOF arm robot) for catching a hammer, a tennis racket, an empty bottle, a partially filled bottle and a cardboard box.

Chapter 36 — Motion for Manipulation Tasks

James Kuffner and Jing Xiao

This chapter serves as an introduction to Part D by giving an overview of motion generation and control strategies in the context of robotic manipulation tasks. Automatic control ranging from the abstract, high-level task specification down to fine-grained feedback at the task interface are considered. Some of the important issues include modeling of the interfaces between the robot and the environment at the different time scales of motion and incorporating sensing and feedback. Manipulation planning is introduced as an extension to the basic motion planning problem, which can be modeled as a hybrid system of continuous configuration spaces arising from the act of grasping and moving parts in the environment. The important example of assembly motion is discussed through the analysis of contact states and compliant motion control. Finally, methods aimed at integrating global planning with state feedback control are summarized.

Mesoscale manipulation: System, modeling, planning and control

Author  David J. Cappelleri et al.

Video ID : 359

This video shows an example of peg-in-hole manipulation on the mesoscale. Three robust motion primitives are introduced, i.e., one-point sticking contact with counterclockwise rotation, two-point contact motion without rotation, and robust rotation. These motion primitives are sequentially executed to accomplish the peg-in-hole manipulation task.

Chapter 34 — Visual Servoing

François Chaumette, Seth Hutchinson and Peter Corke

This chapter introduces visual servo control, using computer vision data in the servo loop to control the motion of a robot. We first describe the basic techniques that are by now well established in the field. We give a general overview of the formulation of the visual servo control problem, and describe the two archetypal visual servo control schemes: image-based and pose-based visual servo control. We then discuss performance and stability issues that pertain to these two schemes, motivating advanced techniques. Of the many advanced techniques that have been developed, we discuss 2.5-D, hybrid, partitioned, and switched approaches. Having covered a variety of control schemes, we deal with target tracking and controlling motion directly in the joint space and extensions to under-actuated ground and aerial robots. We conclude by describing applications of visual servoing in robotics.

PBVS on a 6-DOF robot arm (2)

Author  Francois Chaumette, Seth Hutchinson, Peter Corke

Video ID : 63

This video shows a PBVS on a 6-DOF robot arm with (c*^t_c, theta u) as visual features. It corresponds to the results depicted in Figure 34.10.

Chapter 76 — Evolutionary Robotics

Stefano Nolfi, Josh Bongard, Phil Husbands and Dario Floreano

Evolutionary Robotics is a method for automatically generating artificial brains and morphologies of autonomous robots. This approach is useful both for investigating the design space of robotic applications and for testing scientific hypotheses of biological mechanisms and processes. In this chapter we provide an overview of methods and results of Evolutionary Robotics with robots of different shapes, dimensions, and operation features. We consider both simulated and physical robots with special consideration to the transfer between the two worlds.

Visual navigation of mobile robot with pan-tilt camera

Author  Dario Floreano

Video ID : 36

A mobile robot with a pan-tilt camera is asked to to navigate in a square arena with low walls and located in an office.

Chapter 64 — Rehabilitation and Health Care Robotics

H.F. Machiel Van der Loos, David J. Reinkensmeyer and Eugenio Guglielmelli

The field of rehabilitation robotics considers robotic systems that 1) provide therapy for persons seeking to recover their physical, social, communication, or cognitive function, and/or that 2) assist persons who have a chronic disability to accomplish activities of daily living. This chapter will discuss these two main domains and provide descriptions of the major achievements of the field over its short history and chart out the challenges to come. Specifically, after providing background information on demographics (Sect. 64.1.2) and history (Sect. 64.1.3) of the field, Sect. 64.2 describes physical therapy and exercise training robots, and Sect. 64.3 describes robotic aids for people with disabilities. Section 64.4 then presents recent advances in smart prostheses and orthoses that are related to rehabilitation robotics. Finally, Sect. 64.5 provides an overview of recent work in diagnosis and monitoring for rehabilitation as well as other health-care issues. The reader is referred to Chap. 73 for cognitive rehabilitation robotics and to Chap. 65 for robotic smart home technologies, which are often considered assistive technologies for persons with disabilities. At the conclusion of the present chapter, the reader will be familiar with the history of rehabilitation robotics and its primary accomplishments, and will understand the challenges the field may face in the future as it seeks to improve health care and the well being of persons with disabilities.

BONES and SUE exoskeletons for robotic therapy

Author  Julius Klein, Steve Spencer, James Allington, Marie-Helene Milot, Jim Bobrow, David Reinkensmeyer

Video ID : 498

BONES is a 5-DOF, pneumatic robot developed at the University of California at Irvine for naturalistic arm training after stroke. It incorporates an assistance-as-needed algorithm that adapts in real time to patient errors during game play by developing a computer model of the patient's weakness as a function of workspace location. The controller incorporates an anti-slacking term. SUE is a 2-DOF pneumatic robot for providing wrist assistance. The video shows a person with a stroke using the device to drive a simulated motor cycle through a simulated Death Valley.

Chapter 6 — Model Identification

John Hollerbach, Wisama Khalil and Maxime Gautier

This chapter discusses how to determine the kinematic parameters and the inertial parameters of robot manipulators. Both instances of model identification are cast into a common framework of least-squares parameter estimation, and are shown to have common numerical issues relating to the identifiability of parameters, adequacy of the measurement sets, and numerical robustness. These discussions are generic to any parameter estimation problem, and can be applied in other contexts.

For kinematic calibration, the main aim is to identify the geometric Denavit–Hartenberg (DH) parameters, although joint-based parameters relating to the sensing and transmission elements can also be identified. Endpoint sensing or endpoint constraints can provide equivalent calibration equations. By casting all calibration methods as closed-loop calibration, the calibration index categorizes methods in terms of how many equations per pose are generated.

Inertial parameters may be estimated through the execution of a trajectory while sensing one or more components of force/torque at a joint. Load estimation of a handheld object is simplest because of full mobility and full wrist force-torque sensing. For link inertial parameter estimation, restricted mobility of links nearer the base as well as sensing only the joint torque means that not all inertial parameters can be identified. Those that can be identified are those that affect joint torque, although they may appear in complicated linear combinations.

Calibration and accuracy validation of a FANUC LR Mate 200iC industrial robot

Author  Ilian Bonev

Video ID : 430

This video shows excerpts from the process of calibrating a FANUC LR Mate 200iC industrial robot using two different methods. In the first method, the position of one of three points on the robot end-effector is measured using a FARO laser tracker in 50 specially selected robot configurations (not shown in the video). Then, the robot parameters are identified. Next, the position of one of the three points on the robot's end-effector is measured using the laser tracker in 10,000 completely arbitrary robot configurations. The mean positioning error after calibration was found to be 0.156 mm, the standard deviation (std) 0.067 mm, the mean+3*std 0.356 mm, and the maximum 0.490 mm. In the second method, the complete pose (position and orientation) of the robot end-effector is measured in about 60 robot configurations using an innovative method based on Renishaw's telescoping ballbar. Then, the robot parameters are identified. Next, the position of one of the three points on the robot's end-effector is measured using the laser tracker in 10,000 completely arbitrary robot configurations. The mean position error after calibration was found to be 0.479 mm, the standard deviation (std) 0.214 mm, and the maximum 1.039 mm. However, if we limit the zone for validations, the accuracy of the robot is much better. The second calibration method is less efficient but relies on a piece of equipment that costs only $12,000 (only one tenth the cost of a laser tracker).

Chapter 72 — Social Robotics

Cynthia Breazeal, Kerstin Dautenhahn and Takayuki Kanda

This chapter surveys some of the principal research trends in Social Robotics and its application to human–robot interaction (HRI). Social (or Sociable) robots are designed to interact with people in a natural, interpersonal manner – often to achieve positive outcomes in diverse applications such as education, health, quality of life, entertainment, communication, and tasks requiring collaborative teamwork. The long-term goal of creating social robots that are competent and capable partners for people is quite a challenging task. They will need to be able to communicate naturally with people using both verbal and nonverbal signals. They will need to engage us not only on a cognitive level, but on an emotional level as well in order to provide effective social and task-related support to people. They will need a wide range of socialcognitive skills and a theory of other minds to understand human behavior, and to be intuitively understood by people. A deep understanding of human intelligence and behavior across multiple dimensions (i. e., cognitive, affective, physical, social, etc.) is necessary in order to design robots that can successfully play a beneficial role in the daily lives of people. This requires a multidisciplinary approach where the design of social robot technologies and methodologies are informed by robotics, artificial intelligence, psychology, neuroscience, human factors, design, anthropology, and more.

Explaining a typical session with Sunflower as a home companion in the Robot House

Author  Kerstin Dautenhahn

Video ID : 221

The video illustrates and explains one of the final showcases of the European project LIREC ( in the University of Hertfordshire Robot House. The Sunflower robot, developed at UH, provides cognitive and physical assistance in a home scenario. In the video, one of the researchers, Dag Syrdal, explains a typical session in long-term evaluation studies in the Robot House. Sunflower has access to a network of smart sensors in the Robot House. The video also illustrates the concept of migration (moving of the robot's mind/AI to a differently embodied system).

Chapter 53 — Multiple Mobile Robot Systems

Lynne E. Parker, Daniela Rus and Gaurav S. Sukhatme

Within the context of multiple mobile, and networked robot systems, this chapter explores the current state of the art. After a brief introduction, we first examine architectures for multirobot cooperation, exploring the alternative approaches that have been developed. Next, we explore communications issues and their impact on multirobot teams in Sect. 53.3, followed by a discussion of networked mobile robots in Sect. 53.4. Following this we discuss swarm robot systems in Sect. 53.5 and modular robot systems in Sect. 53.6. While swarm and modular systems typically assume large numbers of homogeneous robots, other types of multirobot systems include heterogeneous robots. We therefore next discuss heterogeneity in cooperative robot teams in Sect. 53.7. Once robot teams allow for individual heterogeneity, issues of task allocation become important; Sect. 53.8 therefore discusses common approaches to task allocation. Section 53.9 discusses the challenges of multirobot learning, and some representative approaches. We outline some of the typical application domains which serve as test beds for multirobot systems research in Sect. 53.10. Finally, we conclude in Sect. 53.11 with some summary remarks and suggestions for further reading.

Reconfigurable multi-agents with distributed sensing for robust mobile robots

Author  Robin Murphy

Video ID : 206

In marsupial teams, a mother robot carries one or more daughter robots. This video shows that a mother robot can opportunistically treat daughter robots as surrogate sensors in order to autonomously reconfigure herself to recover from sensor failures.

Chapter 14 — AI Reasoning Methods for Robotics

Michael Beetz, Raja Chatila, Joachim Hertzberg and Federico Pecora

Artificial intelligence (AI) reasoning technology involving, e.g., inference, planning, and learning, has a track record with a healthy number of successful applications. So can it be used as a toolbox of methods for autonomous mobile robots? Not necessarily, as reasoning on a mobile robot about its dynamic, partially known environment may differ substantially from that in knowledge-based pure software systems, where most of the named successes have been registered. Moreover, recent knowledge about the robot’s environment cannot be given a priori, but needs to be updated from sensor data, involving challenging problems of symbol grounding and knowledge base change. This chapter sketches the main roboticsrelevant topics of symbol-based AI reasoning. Basic methods of knowledge representation and inference are described in general, covering both logicand probability-based approaches. The chapter first gives a motivation by example, to what extent symbolic reasoning has the potential of helping robots perform in the first place. Then (Sect. 14.2), we sketch the landscape of representation languages available for the endeavor. After that (Sect. 14.3), we present approaches and results for several types of practical, robotics-related reasoning tasks, with an emphasis on temporal and spatial reasoning. Plan-based robot control is described in some more detail in Sect. 14.4. Section 14.5 concludes.

SHAKEY: Experimentation in robot learning and planning (1969)

Author  Peter Hart, Nils Nilsson

Video ID : 704

SRI's robot Shakey (built 1966-1972) was the first mobile robot that could reason about its surroundings. This 1969 movie provides a good look at how Shakey worked.

Chapter 58 — Robotics in Hazardous Applications

James Trevelyan, William R. Hamel and Sung-Chul Kang

Robotics researchers have worked hard to realize a long-awaited vision: machines that can eliminate the need for people to work in hazardous environments. Chapter 60 is framed by the vision of disaster response: search and rescue robots carrying people from burning buildings or tunneling through collapsed rock falls to reach trapped miners. In this chapter we review tangible progress towards robots that perform routine work in places too dangerous for humans. Researchers still have many challenges ahead of them but there has been remarkable progress in some areas. Hazardous environments present special challenges for the accomplishment of desired tasks depending on the nature and magnitude of the hazards. Hazards may be present in the form of radiation, toxic contamination, falling objects or potential explosions. Technology that specialized engineering companies can develop and sell without active help from researchers marks the frontier of commercial feasibility. Just inside this border lie teleoperated robots for explosive ordnance disposal (EOD) and for underwater engineering work. Even with the typical tenfold disadvantage in manipulation performance imposed by the limits of today’s telepresence and teleoperation technology, in terms of human dexterity and speed, robots often can offer a more cost-effective solution. However, most routine applications in hazardous environments still lie far beyond the feasibility frontier. Fire fighting, remediating nuclear contamination, reactor decommissioning, tunneling, underwater engineering, underground mining and clearance of landmines and unexploded ordnance still present many unsolved problems.

Radioactive material handling 1954

Author  James P. Trevelyan

Video ID : 587

This clip shows the use of a remotely-operated arm to protect a worker from nuclear radiation, taken from the 1954 film sponsored by General Electric, The Atom Goes to Sea 1954. The scene is at the Knolls Atomic Power Laboratory in Schenectady, New York. The entire film is available from the Internet Archive.