Word embedding methods provide a simple and practical representation of lexical semantics that can be learned in an unsupervised fashion from unannotated corpora. The fact that these methods discover meaning representations automatically makes them attractive as a research tool in investigations where our goal is to discover semantic shifts in diachronic corpora. However, most previous applications of word embeddings in investigations of semantic change have been limited because the common types of word embeddings do not distinguish between different senses of words, which makes the results harder to interpret. In this talk, we describe a number of extensions of diachronic word embedding models that allow them to express sense distinctions, and describe some preliminary experiments where we apply these models in diachronic English corpora.