2021 - ACM - Learning Dynamic Embeddings for Temporal Knowledge Graphs

[VaghehDashti]

Title: Learning Dynamic Embeddings for Temporal Knowledge Graphs Year: 2021 Venue: ACM

Introduction: A Knowledge Graph (KG) is a collection of factual triplets, with each triplet consisting of a subject entity s, an object entity o and their relation r and denoted by (s, r, o). Knowledge graphs are usually dynamic with new triplets being added into the graphs or old triplets being removed from the graphs. Moreover, new entities and types of relations emerge in the KGs over time, which can also result in challenges of modelling KGs. They refer to such knowledge graphs with dynamic changes over entities and relations as Temporal Knowledge Graphs (TKGs) and denote each fact in a TKG by a quadruple (s, r, o, t ) that occur at a specific point in time t. This paper provides a novel method to derive dynamic embeddings for temporal knowledge graphs in a streaming scenario which enables the model to extrapolate future entities and relations.

Main problem: How to learn dynamic embeddings of entities and their relations for a temporal knowledge graph?

Example: Albert Einstein worked in different organizations and had multiple relations with other entities from 1912 to 1955, resulting in the semantics of the entity Albert Einstein shifts over time. Hence, modelling temporal information in KGs is crucial to understand how the knowledge evolves over time to answer questions like "Which university will Albert Einstein work at next year?", and "Which award is Albert Einstein likely to win next time?".

Input: Knowledge graph data in a streaming scenario, where each triplet consists of two entities and their relation arriving sequentially over time. For instance, Einstein works at ETH Zurich between 1912 and 1914 which results in three triplets (one for each year).

Output: Low-dimensional representations of entities and relations in the same semantic space.

Motivation: Capturing how entities and their relationship evolve through time and inferring embeddings for unseen entities and relations.

Previous works and their gaps:

• Some of the previous works have focused on static knowledge graphs, which cannot capture the rich dynamics of temporal knowledge graphs.
• Some works can only predict missing facts based on an interpolation problem formulation and therefore cannot predict future facts.
• Some of the more recent works that utilize RNNs cannot model multiple simultaneous events, i.e., they only model one relation between subject and object at each time step, which is incongruous with the real world.
• In summary, most of them cannot jointly and explicitly capture the evolution of entities and their relations.

Contribution of this paper: They have proposed a novel method to learn high-quality dynamic embeddings in TKGs under the streaming scenario. The model is highly interpretable due to the representation of entities and their relations in a Gaussian distribution. To model the latent representations of entities changing over time as entities make relationships with each other, they define a transition distribution for them.

Proposed Method: Dynamic Bayesian Knowledge Graphs Embedding

Experiments: Wikidata12k and YAGO11k dataset: DBKGE-online consistently outperforms the static and temporal baseline methods for all different time stamps. They also experiment with the size of embeddings: always better performance than previous works, especially as the size goes above 100.

Code: Not provided.

Gaps of this work: This paper is state-of-the-art in the temporal knowledge graph embedding making it hard to find gaps.

[hosseinfani]

@VaghehDashti this is an instance of a good summary. thanks.