2nd CALL FOR PARTICIPATION

The One-Minute Gradual-Empathy Prediction (OMG-Empathy) Competition

held in partnership with the IEEE International Conference on Automatic Face and Gesture Recognition 2019 in Lille, France.

I. Aim and Scope

The ability to perceive, understand and respond to social interactions in a human-like manner is one of the most desired capabilities in artificial agents, particularly social robots. These skills are highly complex and require a focus on several different aspects of research, including affective understanding. An agent which is able to recognize, understand and, most importantly, adapt to different human affective behaviors can increase its own social capabilities by being able to interact and communicate in a natural way.

Emotional expression perception and categorization are extremely popular in the affective computing community. However, the inclusion of emotions in the decision-making process of an agent is not considered in most of the research in this field. To treat emotion expressions as the final goal, although necessary, reduces the usability of such solutions in more complex scenarios. To create a general affective model to be used as a modulator for learning different cognitive tasks, such as modeling intrinsic motivation, creativity, dialog processing, grounded learning, and human-level communication, only emotion perception cannot be the pivotal focus. The integration of perception with intrinsic concepts of emotional understanding, such as a dynamic and evolving mood and affective memory, is required to model the necessary complexity of an interaction and realize adaptability in an agent's social behavior.

Such models are most necessary for the development of real-world social systems, which would communicate and interact with humans in a natural way on a day-to-day basis. This could become the next goal for research on Human-Robot Interaction (HRI) and could be an essential part of the next generation of social robots.

For this challenge, we designed, collected and annotated a novel corpus based on human-human interaction. This novel corpus builds on top of the experience we gathered while organizing the OMG-Emotion Recognition Challenge, making use of state-of-the-art frameworks for data collection and annotation.

The One-Minute Gradual Empathy datasets (OMG-Empathy) contain multi-modal recordings of different individuals discussing predefined topics. One of them, the actor, shares a story about themselves while the other, the listener, reacts to it emotionally. We annotated each interaction based on the listener's own assessment of how they felt while the interaction was taking place.

We encourage the participants to propose state-of-the-art solutions not only based on deep, recurrent and self-organizing neural networks but also traditional methods for feature representation and data processing. We also enforce that the use of contextual information, as well as personalized solutions for empathy assessment, will be extremely important for the development of competitive solutions.

II. Competition Tracks

We let available for the challenge a pre-defined set of training, validation and testing samples. We separate our samples based on each story: 4 stories for training, 1 for validation and 3 for testing. Each story sample is composed of 10 videos with interactions, one for each listener. Although using the same training, validation and testing data split, we propose two tracks which will measure different aspects of self-assessed empathy:

The Personalized Empathy track, where each team must predict the empathy of a specific person. We will evaluate the ability of proposed models to learn the empathic behavior of each of the subjects over a newly perceived story. We encourage the teams to develop models which take into consideration the individual behavior of each subject in the training data.

The Generalized Empathy track, where the teams must predict the general behavior of all the participants over each story. We will measure the performance of the proposed models to learn a general empathic measure for each of the stories individually. We encourage the proposed models to take into consideration the aggregated behavior of all the participants for each story and to generalize this behavior in a newly perceived story.

The training and validation samples will be given to the participants at the beginning of the challenge together with all the associated labels. The test set will be given to the participants without the associated labels. The team`s predictions on the test set will be used to calculate the final metrics of the challenge.

III. How to Participate

To participate in the challenge, please send us an email to barros @ informatik.uni-hamburg.de with the title "OMG-Empathy Team Registration". This e-mail must contain the following information: 

Team Name

Team Members

Affiliation

Participating tracks

We split the corpus into three subsets: training, validation, and testing. The participants will receive the training and validation sets, together with the associated annotations once they subscribe to the challenge. The subscription will be done via e-mail. Each participating team must consist of 1 to 5 participants and must agree to use the data only for scientific purposes. Each team can choose to take part in one or both the tracks.

After the training period is over, the testing set will be released without the associated annotations.

Each team must submit, via e-mail, their final predictions as a .csv file for each video on the test set. Together with the final submission, each team must send a short 2-4 pages paper describing their solution published on Arxiv and the link for a GitHub page to their solution. If a team fails to submit any of these items, their submission will be invalidated. Each team can submit 3 complete submissions for each track. 

IV. Important Dates

25th of September 2018 - Opening of the Challenge - Team registrations begin

1st of October 2018 - Training/validation data and annotation available

3rd of December 2018 - Test data release

5th of December 2018 - Final submission (Results and code)

7th of December 2018 - Final submission (Paper)

10th of December 2018 - Announcement of the winners

V. Organization

Pablo Barros, University of Hamburg, Germany

Nikhil Churamani, University of Cambridge, United Kingdom

Angelica Lim, Simon Fraser University, Canada

Stefan Wermter, Hamburg University, Germany