Large knowledge graphs have become a key asset for search engines and other use cases. They are partly based on automatically extracting structured information from web contents and other texts, using a variety of pattern-matching and machine-learning methods. The semantically organized machine knowledge can be harnessed to better interpret text in news, social media and web tables, contributing to question answering, natural language processing (NLP) and data analytics.

A recent trend that has revolutionized NLP is to capture knowledge latently by billions of parameters of language models, learned at scale from huge text collections in a self-supervised manner. These pre-trained models form the basis of fine-tuning machine-learning solutions for tasks that involve both input texts and broad world knowledge, such as question answering, commonsense reasoning and human-computer conversations.

This talk reviews these advances and discusses lessons learned and limitations (see http://dx.doi.org/10.1561/1900000064 for a survey). Moreover, the talk identifies open challenges and new research opportunities. In particular, it discusses potential synergies of knowledge graphs and language models.