NARECA

Summary of the project proposal

One of the real challenges for research in artificial intelligence is to enable humans to interact with the machine, without having to make specific changes in the application and using all the means necessary for optimal communication. The need to adapt the machine to human behavior becomes more challenging when the user is a child, with cognitive and communicative abilities still developing. Designing such an interface requires to improve existing models of dialogue, as well as their use in an Embodied Conversational Agent (en: ECA, fr: ACA). However, it is currently not possible to design a generic ECA without taking into account several scientific limits (speech processing, prosody, emotion, nonverbal behavior, vocabulary size, linguistic development, etc.) .

The objective of the NARECA project is to optimize the modeling of dialogue interactions and use these models in an ECA. The scientific experiments envisaged are linked to extraction of multi-modal and affective interaction models on real dialogues (both methodology and tools). Our approach wants to identify recurring patterns of behavior in dialogues, annotate them automatically using NLP tools and then apply extraction patterns and data mining techniques on these annotations. The product of this study will be a software platform for analysis of dialogues, as well as demonstrative application integrating the dialogue patterns collected: an Affective Narrative ECA. The expected benefits are multidisciplinary and the evaluation of our approach will be through this application.

NARECA follows the preliminary results obtained during the ACAMODIA project (2010-2012) funded by the CNRS as a PEPS INS2I-INSHS.

Context and objectives

The Proposed Scenario (en: “Babar and The Missing Crown Affair”, fr: “Babar et l’affaire de la couronne”):

Narrator: They’ll find the crown, don’t worry. So, they hide, they seek who could have taken the crown.
Child: It is there, it is there the crown!
Narrator: They suspect other people: Cornelius , Celeste , the old lady … Who could take the crown ?
Child: The crown, it is there !
Narrator: You think ?
Child: Yes !

In 2010, a student enrolled in primary school has spent more time in front of television than with his teacher (average 864 hours against 956 hours watching TV), knowing that daily exposure of two hours of television multiplies by three the risk of language development delay and for one year the risks are multiplied by 6 (Chonchaiya W. et Prukasananonda, C. 2008). The problem lies in the drastic reduction of interactions, too often replaced by television. As Serge Tisseron said, “TV teaches nothing to the child because it is never interactive” (Tisseron S. and B. Stiegler 2010). It is therefore not a problem of television itself, but the time passed in front of these screens is non-interactive. The problem might be partially avoided if the child could interact with these environments. In recent years, new human-machine interfaces are developed as Embodied Conversational Agents (ECA). The dialogue with this new type of interlocutors, by replacing the hours spent watching television, should be beneficial to social, language and cognitive development of children, as previous work already shows (Ryokai, Vaucelle, et Cassell 2002; Ryokai, Vaucelle, et Cassell 2003). An interactive storytelling ECA increases the interest of children to the virtual world and has the advantage of developing the early socio-cognitive learning skills (Le Sourn-Bissaoui S. et Hooge-Lespagnol F. 2006; Chanoni 2009).

In general, with the increase usage of virtual agents in our daily environment (video games, assistant agents on the web applications, etc..) , human-agent interaction becomes more effective, but still very far from the richness of human-human interaction. Their use with children whose socio-cognitive skills are still developing is potentially problematic. An ECA brings a solution to this problem by offering more enjoyable and similar to human-human interactions interactions. However, the development of good quality ECA remains challenging because of the implementation issues (prosody, emotion, nonverbal behavior, etc..). In particular, the effective management of dialogue, taking into account multi-modal and affective characteristics, remains a very difficult task (Swartout et al. 2006).

The NARECA project focuses on interactive storytelling interaction between an affective ECA and a child. To do this, we propose to tackle two major scientific problems: 1) the extraction of (semi-) automatic dialogue models from real corpus; 2) the integration of a real dialogue models in the ECA. To solve the first problem, we propose to identify behavioral patterns recurring in narrative dialogues annotated automatically, by applying learning and data mining methods for the semi-automatic extraction of knowledge. The extracted dialogue models, specific to a given task will then be integrated into an ECA taking into account both the application goal desired by dialogue (dialogue planning of the narration) and the social interactions, in the form of dialogue games (Hulstijn 2000; Maudet N. 2002). The emotional narrative ECA will be presented as a final demo.

NARECA is a multidisciplinary fundamental research project, having both research and business partners. Its objective is to break the traditional approach used to model human-computer interactions and the implementation of the resulting models into an emotional ECA.