issues
search
flrngel
/
understanding-ai
personal repository
36
stars
6
forks
source link
A Quantum Many-body Wave Function Inspired Language Modeling Approach
#22
Open
flrngel
opened
5 years ago
flrngel
commented
5 years ago
Abstract
quantum probability theory
quantum-inspired LMs have two limitations
not taken into account interaction among the words with multiple meanings
lacking theoretical foundation accounting for effective training parameters
QMWF
can adopt the tensor product to model which interactions mong words
1. Introduction
two LM approaches
classical
increase the number of parameters to be estimated for compound dependencies
quantum inspired
estimates density matrix
encodes both single words and compound words (
QLM
)
QLM
and
NNQLM
end-toend neural network structure
are not modeled to complex interaction among words with multiple meanings
no principled manner
QWMF
the wave function can model the interaction among many spinful particles
convolutional neural network architecture can be mathematically derived in quantum-inspired language modeling approach
outperforms quantum LM counterparts (QLM, NNQLM)
In this paper,
propose Quantum Many-body Wave Function based Language Modeling approach
able to represent complex interaction among words
show fundamental connection between QMWF and CNN
performance checking with QA datasets
2. Quantum Preliminaries
2.1. Basic notation and concepts
2.2. Quantum Many-body Wave Functions
3. Quantum many-body wave function inspired language modeling
3.1. Basic intuitions and architecture
QMWF representation can model probability distribution of compound meanings
depends on basis vectors
3.2. Language representation and projection via many-body wave function
Local representation by product state
Global representation for all possible compound meanings
3.3. Production realized by convolutional neural network
6. Experiments
My notes
Can we think that QMWF is generalized version of attention mechanism?
sum of
a_i^2
is 1
M = 1?
Abstract
1. Introduction
In this paper,
2. Quantum Preliminaries
2.1. Basic notation and concepts
2.2. Quantum Many-body Wave Functions
3. Quantum many-body wave function inspired language modeling
3.1. Basic intuitions and architecture
3.2. Language representation and projection via many-body wave function
3.3. Production realized by convolutional neural network
6. Experiments
My notes