To date, statistical approaches to dialogue management based on Reinforcement Learning have focused on slot-filling spoken dialogue systems, largely ignoring other types of dialogue. The goal of this project is to extend this line of research to allow automated dialogue policy creation for multiple types of dialogue, including question-answering and negotiation.