Cambridge University Press
978-0-521-84511-3 - Imitation and Social Learning in Robots, Humans and Animals - Behavioural, Social and Communicative Dimensions - by Chrystopher L. Nehaniv and Kerstin Dautenhahn
Frontmatter/Prelims



Imitation and Social Learning in Robots, Humans
and Animals




Mechanisms of imitation and social matching play a fundamental role in development, communication, interaction, learning and culture. Their investigation in different agents (animals, humans and robots) has significantly influenced our understanding of the nature and origins of social intelligence. Whilst such issues have traditionally been studied in areas such as psychology, biology and ethology, it has become increasingly recognized that a ‘constructive approach’ towards imitation and social learning via the synthesis of artificial agents can provide important insights into mechanisms and create artifacts that can be instructed and taught by imitation, demonstration and social interaction rather than by explicit programming. This book studies increasingly sophisticated models and mechanisms of social matching behaviour and marks an important step towards the development of an interdisciplinary research field, consolidating and providing a valuable reference for the increasing number of researchers in the field of imitation and social learning in robots, humans and animals.

CHRYSTOPHER L. NEHANIV is Research Professor of Mathematical and Evolutionary Computer Sciences in the School of Computer Science at the University of Hertfordshire, where he works with the Adaptive Systems, Algorithms and BioComputation Research Groups. He is the Director of the UK EPSRC Network on Evolvability in Biological and Software Systems and an Associate Editor of BioSystems: Journal of Biological and Information Processing Sciences and Interaction Studies: Social Behaviour and Communication in Biological and Artificial Systems.

KERSTIN DAUTENHAHN is Research Professor of Artificial Intelligence in the School of Computer Science at the University of Hertfordshire, where she is a coordinator of the Adaptive Systems Research Group. Her research interests include social learning, human–robot interaction, social robotics, narrative and robotic-assisted therapy for children with autism. She is the Editor-in-Chief of Interaction Studies: Social Behaviour and Communication in Biological and Artificial Systems and the general chair of the IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2006).



Imitation and Social Learning
in Robots, Humans and
Animals

Behavioural, Social and Communicative
Dimensions

Edited by

Chrystopher L. Nehaniv and Kerstin Dautenhahn



CAMBRIDGE UNIVERSITY PRESS
Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, São Paulo

Cambridge University Press
The Edinburgh Building, Cambridge CB2 8RU, UK

Published in the United States of America by Cambridge University Press, New York

www.cambridge.org
Information on this title: www.cambridge.org/9780521845113

© Cambridge University Press 2007

This publication is in copyright. Subject to statutory exception
and to the provisions of relevant collective licensing agreements,
no reproduction of any part may take place without
the written permission of Cambridge University Press.

First published 2007

Printed in the United Kingdom at the University Press, Cambridge

A catalogue record for this publication is available from the British Library

Library of Congress cataloguing in publication data
Imitation and social learning in robots, humans and animals: behavioural, social and
communicative dimensions / edited by Chrystopher L. Nehaniv and Kerstin Dautenhahn.
p. cm.
Includes bibliographical references and index.
ISBN-13: 978-0-521-84511-3 (hardback)
1. Robotics. 2. Robots – Control systems. I. Nehaniv, Chrystopher L., 1963–.
II. Dautenhahn, Kerstin.
TJ211.I38 2006
629.8′92 – dc22
2206023587

ISBN 978-0-521-84511-3 hardback




Cambridge University Press has no responsibility for the persistence or accuracy of URLs
for external or third-party internet websites referred to in this publication, and does not
guarantee that any content on such websites is, or will remain, accurate or appropriate.





To our parents,
Chrystyna, Bohdan, Annelie and Claus-Peter





Contents




  List of plates page x
  List of figures xii
  List of tables xvii
  List of contributors xiii
 
  Introduction: the constructive interdisciplinary
viewpoint for understanding mechanisms and models
of imitation and social learning		 1
  CHRYSTOPHER L. NEHANIV AND KERSTIN
DAUTENHAHN
Part I   Correspondence problems and mechanisms 19
1   Imitation: thoughts about theories 23
  GEOFFREY BIRD AND CECILIA HEYES
2   Nine billion correspondence problems 35
  CHRYSTOPHER L. NEHANIV
3   Challenges and issues faced in building a framework for
conducting research in learning from observation		 47
  DARRIN BENTIVEGNA, CHRISTOPHER ATKESON
AND GORDON CHENG
 
Part II   Mirroring and ‘mind-reading’ 67
4   A neural architecture for imitation and
intentional relations		 71
  MARCO IACOBONI, JONAS KAPLAN
AND STEPHEN WILSON
5   Simulation theory of understanding others: a robotics
perspective		 89
  YIANNIS DEMIRIS AND MATTHEW JOHNSON
6   Mirrors and matchings: imitation from the perspective
of mirror-self-recognition, and the parietal region’s
involvement in both		 103
  ROBERT W. MITCHELL
 
Part III   What to imitate? 131
7   The question of ‘what to imitate’: inferring goals and
intentions from demonstrations		 135
  MALINDA CARPENTER AND JOSEP CALL
8   Learning of gestures by imitation in a humanoid robot 153
  SYLVAIN CALINON AND AUDE BILLARD
9   The dynamic emergence of categories through imitation 179
  TONY BELPAEME, BART DE BOER AND BART JANSEN
 
Part IV   Development and embodiment 195
10   Copying strategies by people with autistic spectrum
disorder: why only imitation leads to social cognitive
development		 199
  JUSTIN H. G. WILLIAMS
11   A Bayesian model of imitation in infants and robots 217
  RAJESH P. N. RAO, AARON P. SHON AND ANDREW N.
MELTZOFF
12   Solving the correspondence problem in robotic imitation
across embodiments: synchrony, perception and culture
in artifacts		 249
  ARIS ALISSANDRAKIS, CHRYSTOPHER L. NEHANIV
AND KERSTIN DAUTENHAHN
 
Part V   Synchrony and turn-taking as communicative
mechanisms		 275
13   How to build an imitator 279
  ARNAUD REVEL AND JACQUELINE NADEL
14   Simulated turn-taking and development of styles of
motion		 301
  TAKASHI IKEGAMI AND HIROKI IIZUKA
15   Bullying behaviour, empathy and imitation: an
attempted synthesis		 323
  KERSTIN DAUTENHAHN, SARAH N. WOODS AND
CHRISTINA KAOURI
 
Part VI   Why imitate? – Motivations 341
16   Multiple motivations for imitation in infancy 343
  MARK NIELSEN AND VIRGINIA SLAUGHTER
17   The progress drive hypothesis: an interpretation
of early imitation		 361
  FRéDéRIC KAPLAN AND PIERRE-YVES OUDEYER
 
Part VII   Social feedback 379
18   Training behavior by imitation: from parrots
to people . . . to robots?		 383
  IRENE M. PEPPERBERG AND DIANE V. SHERMAN
19   Task learning through imitation and human–robot
interaction		 407
  MONICA N. NICOLESCU AND MAJA J. MATARI
 
Part VIII   The ecological context 425
20   Emulation learning: the integration of technical
and social cognition		 427
  LUDWIG HUBER
21   Mimicry as deceptive resemblance: beyond
the one-trick ponies		 441
  MARK D. NORMAN AND TOM TREGENZA
  Index 455



Plates




1. Figure 20.1 Curious keas (Nestor notabilis) in Mt Cook
National Park, New Zealand.

(Photos by D. Werdenich and B. Voelkl)

2. Figure 21.1 Leaf mimicry.

  a. Australian Leaf-wing butterfly, Dolleschallia bisaltide;

  b. Cockatoo waspfish, Ablabys taenianotus;

  c. Juvenile round batfish, Platax orbicularis;

  d. Broadclub cuttlefish, Sepia latimanus, ‘impersonating’
dead mangrove leaf;

  e. Same individual in resting colour pattern.

(Photos by M. Norman)

3. Figure 21.2 Camouflage and deceptive resemblance in
  cephalopods.

  a. Octopus sp. 5 (Norman, 2000) active in rock pools;

  b. Same individual moments later in camouflage;

  c. Hairy octopus (Octopus sp. 16 (Norman, 2000)) showing branching skin sculpture;

  d. Snail mimic octopus (Octopus sp. 6 (Gosliner et al., 1996));

  e. Giant cuttlefish, Sepia apama – sneaker male (centre) impersonating female amongst breeding pair (male on right)

(Photos a, b & e by M. Norman; c by Becca Saunders and d by M. Severns)

4. Figure 21.3 Mimic octopus (Octopus sp. 19 (Norman, 2000)) and models.

  a. Sentinel state in mouth of burrow;

  b. Normal foraging colour pattern;

  c. Flatfish mimicry;

  d. Flatfish model, banded sole (Zebrias sp.);

  e. Lionfish mimicry;

  f. Lionfish model (Pterois sp.);

  g. Sea snake mimicry;

  h. Sea snake model, banded sea snake (Laticauda sp.).

(Photos a, b, d, e & f by M. Norman and c, g & h by R. Steene)

5. Figure 3.3 The software air hockey game on the left and air hockey playing with a humanoid robot on the right.

6. Figure 3.5 Six manually defined configurations are used to interpolate the body configuration needed to place the paddle at a specific location.

7. Figure 7.1a and b The demonstrator performing the action (a) in a happy, satisfied manner and (b) in a frustrated, dissatisfied manner in Behne and colleagues’ (2006b) study.

8. Figure 7.2a and b The demonstrator (a) attending and (b) not attending to her action in Behne and colleagues’ (2006a) study.



Figures




1.1   Schematic representation of the associative sequence
learning theory of imitation learning.		 page 26
3.1   Three part framework for learning using primitives. 50
3.2   Software and hardware marble maze environments. 51
3.3   Software air hockey game and playing air hockey with
humanoid robot.		 51
3.4   Primitives being explored in the marble maze. 52
3.5   Manually defined configurations of robot used to
interpolate configuration for paddle
placement.		 53
3.6   Raw observed data, wall contact and recognized
primitives.		 55
3.7   Performance on marble maze. 56
3.8   Paddle trajectories and desired paths in shot maneuvers. 58
4.1   Time series in a human premotor area for imitation and
action observation.		 73
4.2   Anatomical locations of brain regions implicated in the
neural architecture for imitation.		 76
4.3   The neural architecture for imitation mapped onto the
functional elements of intentional relations theory.		 78
5.1   Inverse and forward models paired for action
recognition.		 93
5.2   The robot imitating picking up an object. 95
5.3   Arrangement of primitives in graphed inverse models:
“Pick up object” and “Place object”.		 95
5.4   Confidences of inverse models while the robot learns to
pick up an object.		 98
7.1   The demonstrator performing the action in a happy,
satisfied manner and in a frustrated, dissatisfied manner
in Behne and colleagues’ study.		 141
7.2   The demonstrator attending and not attending to her
action in Behne and colleagues’ study.		 141
7.3   Components and processes of a flexible imitation
system.		 145
8.1   Demonstration and reproduction of different tasks. 155
8.2   Demonstration and reproduction of drawing the three
stylized alphabet letters A, B and C.		 156
8.3   Schematic of the sensory-motor flow. 158
8.4   Encoding of the hand path in Cartesian space and joint
angles trajectories in a HMM.		 160
8.5   BIC criterion is used to determine the optimal number
of states of the HMM required to encode the data.		 161
8.6   Example of the retrieval process. 163
8.7   Function used to transform a standard deviation to a
weight factor w in [0, 1].		 164
8.8   Four demonstrations of the drinking gesture and the
knocking gesture.		 165
8.9   Comparison of two metrics to evaluate the quality of a
reproduced trajectory.		 166
8.10   Demonstration, transformation by PCA, encoding in
HMM and retrieval of the two different motions waving
goodbye and drawing letter B.		 168
8.11   Cost function when testing the gestures of the test set
with the different HMMs.		 169
8.12   Byrne’s string parsing imitation
model.		 172
8.13   Heyes and Ray’s associative sequence learning (ASL) model. 173
8.14   Nehaniv and Dautenhahn’s algebraic framework to map
states, effects and/or actions of the demonstrator and
the imitator.		 174
9.1   An agent has categories made up of action
and observation.		 183
9.2   Schematic outline of the imitation game. 184
9.3   Agents use a robot arm to execute actions and a stereo
camera to observe actions.		 185
9.4   Results for 50 000 imitation games between two robots. 189
9.5   Imitative success and number of actions learnt in a
simulation of the imitation game.		 190
10.1   Glass brains showing areas of brain activation during
fMRI that were greater during the imitation condition
than the spatial cue-execution condition.		 209
11.1   Imitative responses in 2- to 3-week-old infants. 220
11.2   A 14-month-old infant imitating the novel action of
touching a panel with the forehead.		 222
11.3   Infants as young as 14 months old can imitate actions
on objects as seen on TV; infants can also perform
deferred imitation based on actions observed on TV the
previous day.		 224
11.4   Infants attributed goals and intentions to human but
not to inanimate device performing unsuccessful act.		 225
11.5   Simulated maze environment and learned forward
model.		 233
11.6   Learned priors and example of successful imitation. 235
11.7   Inferring the intent of the teacher. 236
11.8   Robotic platforms for testing Bayesian imitation models. 238
11.9   Meltzoff and Moore’s AIM model of facial imitation. 240
12.1   The Action Learning via Imitation between
Corresponding Embodiments (ALICE) framework.		 251
12.2   Two CHESSWORLD examples. 254
12.3   The ALICE framework as realized in the CHESSWORLD
testbed.		 255
12.4   Example embodiment of a robotic arm agent. 256
  Different examples of behaviours. 257
12.6   An example of using the metrics to compare actions,
states (before and after) and effects between two agents.		 258
12.7   The ALICE framework as realized in the RABIT
robotic-arm testbed.		 261
12.8   An example of social transmission. 262
12.9   Examples of emerging ‘proto-culture’. 263
12.10   More examples of emerging ‘proto-culture’. 263
12.11   Experiments comparing the use of synchronization. 265
12.12   Experiments comparing using and not using
proprioception.		 267
12.13   Experiments comparing the use of loose perceptual
matching.		 268
12.14   Example of an agent imitating with a changing
embodiment.		 269
12.15   Another example of an agent imitating with a changing
embodiment.		 270
12.16   Yet another example of an agent imitating with a
changing embodiment.		 270
12.17   Qualitative effect of the metric used by an imitating
agent that changes embodiment.		 271
13.1   Simple sensorimotor connections allow the building of
‘reflex’ behaviours.		 281
13.2   Any low-level processing allows the building of
‘perceptive’ information.		 281
13.3   Complete PerAc architecture 283
13.4   Mother–infant interaction. 289
13.5   Temporal delay for holds and discards. 290
13.6   Two PerAc systems S1 and S2 in interaction. 292
13.7   An imitating behaviour emerges. 293
13.8   Child A’s hat has slipped behind his back. Child B seeks
for his hat behind his back although his hat is still on his
head.		 296
13.9   Each system has its own motivations which are only
accessible to the other system via the interaction.		 296
14.1   Schematic view of the mobile agent with two wheels,
and two mobile agents interact to perform turn-taking
behaviour.		 304
14.2   An agent has a three-layered recurrent network. 305
14.3   The fitness value of the best agent as a function of GA
generations.		 307
14.4   Spatial patterns of turn-taking behavior observed in the
simulations.		 309
14.5   Spatial trails of the agents with sequential random
inputs for each 100 steps.		 310
14.6   Prediction performance and turns for an agent after
10 000 GA generations.		 311
14.7   Spatial trails of agents from 3000th and 9000th
generation in interactions with a controlled agent to
constant output.		 312
14.8   Spatial patterns of the original pairs and newly coupled
agents at 7000, 8000 and 10 000 GA generations.		 313
14.9   Adaptability is computed for each agent from
populations A and B, and gradually increases with later
GA generations.		 315
15.1   Sketch of possible connections between imitation,
empathy and bullying behaviour.		 332
16.1   Cumulative percentage of infants showing evidence of
immediate imitation, deferred imitation and synchronic
imitation.		 350
17.1   The critic uses a predictor P and a metapredictor metaP. 365
17.2   The screen problem. 367
17.3   Evolution of the average values of the Noise and Reset
buttons.		 370
17.4   Structure of local experts built during the experiments. 370
17.5   Tree-like structure showing how the child could
progressively organize its sensorimotor space.		 373
19.1   Structure of the inputs/outputs of an abstract and
primitive behaviour.		 409
19.2   Example of a behaviour network. 409
19.3   The human–robot interaction experiments setup. 411
19.4   Structure of the environment and course of
demonstration.		 415
19.5   Evolution of the task representation over three
successive demonstrations.		 416
19.6   Structure of the environment and course of task
execution during practice and feedback.		 417
19.7   Topologies obtained after practice and feedback. 418
20.1   Curious keas (Nestor notabilis). 431
21.1   Leaf mimicry. 444
21.2   Camouflage and deceptive resemblance in cephalopods. 447
21.3   Mimic octopus (Octopus sp. 19 (Norman, 2000)) and
models.		 450



Tables




  2.1  Classes of correspondence problems arising from
particular combinations of granularities and metrics.		 page 38
  10.1  Brain area activation clusters (controls and ASD group) 215
  10.2  Brain area activation clusters (group differences – control
>ASD)		 216
  10.3  Brain area activation clusters (group differences – ASD
>control)		 216
  18.1  Effects of training method 389


Contributors




ARIS ALISSANDRAKIS University of Hertfordshire, Adaptive Systems Research Group, UK

CHRISTOPHER ATKESON ATR Computational Neuroscience Lab- oratories, Department of Humanoid Robotics and Computational Neuroscience, Kyoto, Japan and Carnegie Mellon University, Robotics Institute, Pittsburgh, USA

TONY BELPAEME University of Plymouth, School of Computing, Communications and Electronics, UK

DARRIN BENTIVEGNA ATR Computational Neuroscience Lab- oratories, Department of Humanoid Robotics and Computational Neuroscience, Kyoto, Japan and Computational Brain Project, ICORP, Japan Science and Technology Agency, Kyoto, Japan

AUDE BILLARD Swiss Federal Institute of Technology Lausanne (EPFL), Autonomous Systems Lab, Switzerland

GEOFFREY BIRD Department of Psychology and Institute of Cognitive Neuroscience, University College London, UK

JOSEP CALL Max Planck Institute for Evolutionary Anthropology, Leipzig, Germany

SYLVAIN CALINON Swiss Federal Institute of Technology Lausanne (EPFL), Autonomous Systems Lab, Switzerland

MALINDA CARPENTER Max Planck Institute for Evolutionary Anthropology, Leipzig, Germany

GORDON CHENG ATR Computational Neuroscience Laboratories, Department of Humanoid Robotics and Computational Neuroscience, Kyoto, Japan and Computational Brain Project, ICORP, Japan Science and Technology Agency, Kyoto, Japan

KERSTIN DAUTENHAHN University of Hertfordshire, Adaptive Systems Research Group, UK

BART DE BOER Rijksuniversiteit Groningen, Kunstmatige Intelligentie, the Netherlands

YIANNIS DEMIRIS Imperial College London, Department of Electrical and Electronic Engineering, UK

CECILIA HEYES Department of Psychology and Institute of Cognitive Neuroscience, University College London, UK

LUDWIG HUBER University of Vienna, Institute of Zoology, Austria

MARCO IACOBONI University of California, Department of Psychiatry and Biobehavioral Sciences, Neuropsychiatric Institute and Brain Research Institute, UK

HIROYUKI IIZUKA University of Tokyo, Department of General Systems Sciences, Japan

TAKASHI IKEGAMI University of Tokyo, Department of General Systems Sciences, Japan

BART JANSEN Vrije Universiteit Brussel (VUB), Artificial Intelligence Lab, Belgiam

MATTHEW JOHNSON Imperial College London, Department of Electrical and Electronic Engineering, UK

CHRISTINA KAOURI University of Hertfordshire, Adaptive Systems Research Group, UK

FRéDéRIC KAPLAN Sony Computer Science Laboratory, Paris, France

JONAS KAPLAN University of California, FPR-UCLA Center for Culture, Brain and Development, Department of Psychology, USA

MAJA J. MATARIć University of Southern California, Computer Science Department, USA

ANDREW N. MELTZOFF University of Washington, Institute for Learning and Brain Sciences, Seattle, USA

ROBERT W. MITCHELL Eastern Kentucky University, Department of Psychology, USA

JACQUELINE NADEL CNRS, Group Development and Psychopathology, France

CHRYSTOPHER L. NEHANIV University of Hertfordshire, Adaptive Systems and Algorithms Research Groups, UK

MONICA N. NICOLESCU University of Nevada, Department of Computer Science and Engineering, USA

MARK NIELSEN University of Queensland, School of Psychology, Early Cognitive Development Unit, Australia

MARK D. NORMAN Museum Victoria, Department of Marine Invertebrates, Australia

PIERRE-YVES OUDEYER Sony Computer Science Laboratory, Paris

IRENE M. PEPPERBERG MIT School of Architecture and Planning and Brandeis University, Department of Psychology, USA

RAJESH P. N. RAO University of Washington, Department of Computer Science and Engineering, USA

ARNAUD REVEL CNRS, Group ETIS, France

DIANE V. SHERMAN New-Found Therapies, Inc., USA

AARON P. SHON University of Washington, Department of Computer Science and Engineering, USA

VIRGINIA SLAUGHTER University of Queensland, School of Psychology, Early Cognitive Development Unit, Australia

TOM TREGENZA University of Exeter, School of Biosciences, Centre for Ecology & Conservation, UK

JUSTIN H. G. WILLIAMS University of Aberdeen Medical School, Department of Child Health, UK

STEPHEN WILSON University of California, Department of Psychiatry and Biobehavioral Sciences, Brain Research Institute, USA

SARAH N. WOODS University of Hertfordshire, Adaptive Systems Research Group, UK


© Cambridge University Press