The intrinsic motivation of reinforcement and imitation learning for sequential tasks
La motivation intrinsèque d'apprendre par renforcement et imitation des tâches séquentielles
Résumé
My work in the field of developmental cognitive robotics aims to devise a new domain bridging between reinforcement learning and imitation learning, with a model of the intrinsic motivation for learning agents to learn with guidance from tutors multiple tasks, including sequential tasks. My main contribution has been to propose a common formulation of intrinsic motivation based on empirical progress for a learning agent to choose automatically its learning curriculum by actively choosing its learning strategy for simple or sequential tasks: which task to learn, between autonomous exploration or imitation learning, between low-level actions or task decomposition, between several tutors. The originality is to design a learner that benefits not only passively from data provided by tutors, but to actively choose when to request tutoring and what and whom to ask. The learner is thus more robust to the quality of the tutoring and learns faster with fewer demonstrations.
With my previous advisors, colleagues and my collaborators, we developed the framework of socially guided intrinsic motivation with machine learning algorithms to learn multiple tasks by taking advantage of the generalisability properties of human demonstrations in a passive manner or in an active manner through requests of demonstrations from the best tutor for simple and composing subtasks. The latter relies on a representation of subtask composition proposed for a construction process, which should be refined by representations used for observational processes of analysing human movements and activities of daily living. With the outlook of a language-like communication with the tutor, we investigated the emergence of a symbolic representation of the continuous sensorimotor space and of tasks using intrinsic motivation.
These works proposed within the reinforcement learning framework, a reward function for interacting with tutors for multi-task learning. The formulation of this reward unifies interactive learning, multi-task learning and hierarchical learning through the same empirical competence progress measure as the intrinsic motivation to choose the different aspects its learning strategy.
The socially guided intrinsic motivation framework could be used for intelligent tutoring systems to be applied in socially assistive robotics for a robot coach for physical rehabilitation of low-back pain patients analysing whole body movements, or for a robot coach for ASD children in an imitation game.
The project I wish to develop in cognitive developmental learning pertains both to the reinforcement learning framework of machine learning, to control learning in the perception-action loop of cognitive robotics and the theory of intrinsic motivation from developmental psychology in cognitive science. The objective is to propose new reinforcement learning algorithms for robots to learn by actively interacting with their environment, enabling robots to master control tasks of growing complexity, while extending the current theory of intrinsic motivation towards a unified model of motivations for interacting with tutors. The project relies on two axes : a model of interaction with tutors relying on an emerging representation of tasks by the robot, and a model of motivation of human learners through the use case of a robot coach for physical rehabilitation. This work will enrich the formulation of the motivation for social guidance with various motivational drives of learners to interact with tutors, taking the perspective of communication and emotions. This will also highlight the importance of emerging internal representations for understanding social guidance, thus emphasising the interdependence between autonomous learning and social learning.
Mots clés
Intrinsic motivation
Active imitation learning
Reinforcement leaning
Hierarchical learning
automatic curriculum learning
Active Imitation Learning
Social Guidance
Intrinsic Motivation
Reinforcement Learning
Hierarchical Learning
Human Movement Analysis
Activities of Daily Living
Activity Recognition
Socially Assistive Robotics
Smart Home
Origine | Fichiers produits par l'(les) auteur(s) |
---|