Abstract
The main goal of modeling human conversation is to create agents which can interact with people in both open-ended and goal-oriented scenarios. End-to-end trained neural dialog systems are an important line of research for such generalized dialog models as they do not resort to any situation-specific handcrafting of rules. However, incorporating personalization into such systems is a largely unexplored topic as there are no existing corpora to facilitate such work. In this paper, we present a new dataset of goal-oriented dialogs which are influenced by speaker profiles attached to them. We analyze the shortcomings of an existing end-to-end dialog system based on Memory Networks and propose modifications to the architecture which enable personalization. We also investigate personalization in dialog as a multi-task learning problem, and show that a single model which shares features among various profiles outperforms separate models for each profile.
Personalized dialog systems
The main goal of modelling human conversation is to create agents which can interact with people in both open-ended and goal-oriented scenarios. End-to-end trained neural dialog systems are an important line of research for such generalized dialog models as they do not resort to any situation-specific handcrafting of rules. Modelling personalization into conversations in such agents is important for them to be truly ‘smart’ and to integrate seamlessly into the lives of human beings.
A new dialog dataset
However, the topic of personalization has been largely unexplored by researchers as there are no existing corpora for training dialog systems on conversations that are influenced by the profiles of the speakers involved. We created a new dataset of goal-oriented dialogs with profiles attached to them. Our synthetically generated dialogs were set in the domain of restaurant reservation and were built on top of the bAbI dialog dataset.
In addition to the goal of the original bAbI dialog tasks, the dialog system must leverage a user’s profile information to alter its speech style and make personalized suggestions. The end-goal is to make a restaurant reservation that is personalized to the user’s attributes (dietary preference, favorite food items etc.).
Split memory architecture for Memory Networks
We provided baselines on our tasks using a variety of learning methods and investigated in detail the shortcomings of an end-to-end dialog system based on Memory Networks.
The model was unable to perform compositional reasoning or personalization to solve the tasks, indicating that further work needs to be done on learning methods for these aspects. Towards this goal, we proposed a split memory architecture for that lead to an improvement in overall performance.
Why profile attributes?
We realized that models could deal with profiles explicitly as, (1) a structured set of attributes for each person, or (2) by reasoning over historical conversations with the same person. This paper discusses the first scenario. The second scenario seems more applicable to open-ended dialogs (chit-chat) such as tweets, forum threads etc.
Why synthetic tasks?
The synthetic nature of the bAbI dialog dataset and by extension, our work, makes it easy to construct a perfect handcrafted dialog system. Hence, the goal of such tasks is not to improve the state of the art in the given domains, but to analyze existing end-to-end dialog systems and to study how to model personalization in such frameworks. This work is only an initial step in this direction of research.
Some of the questions we wanted to pose and address were-
- How to build personalization into dialog models?
- How to evaluate a dialog model’s capability to personalize?
- How to obtain real world data for such tasks?
In general, the ML community has had a lot of success in studying new theories and learning methods on toy tasks before applying them to real world data.
The full paper is available on arXiv and our dataset and code for all models is accessable via ParlAI or Github.
