January 1, 2015
Dr Aylett made a significant contribution to the proposal for this Horizon H2020 grant. He was the PI for the project for CereProc. The Noxi database and the(non)verbal Annotator Tool (NOVA) were the outputs of the project together with a downloadable working system which allowed interaction with an Alice character to explore the book - Alice’s Adventures in Wonderland by Lewis Carroll.
The ARIA-VALUSPA (Artificial Retrieval of Information Assistants – Virtual Agents with Linguistic Understanding, Social skills, and Personalised Aspects) project worked on a framework that will allow easy creation of Artificial Retrieval of Information Assistants (ARIAs) that are capable of holding multi-modal social interactions in challenging and unexpected situations.
The system can generate search queries and return the information requested by interacting with humans through virtual characters. These virtual humans were able to sustain an interaction with a user for some time, and react appropriately to the user’s verbal and non-verbal behavior when presenting the requested information and refining search results. Using audio and video signals as input, both verbal and non-verbal components of human communication are captured. Together with an emotive personality model, a sophisticated dialogue management system decides how to respond to a user’s input, be it a spoken sentence, a head nod, or a smile. The ARIA uses special speech synthesizers to create emotionally colored speech and a fully expressive 3D face to create the chosen response. Backchannelling, indicating that the ARIA understood what the user meant, or returning a smile are but a few of the many ways in which it can employ emotionally colored social signals to improve communication.