rpinsler
commented
9 years ago I came up with a solution, but don't know how to properly pass the number of mixing coefficients, M, as a parameter. The code looks as follows:
M = 3
model_gmm = Sequential()
[...]
model_gmm.add(Dense((M+2)*input_dim))
model_gmm.add(Lambda(gmm_activation(M))
model_gmm.compile(loss=gmm_loss(M), optimizer='adam')
def gmm_activation(M):
"""
GMM-like activation function.
Assumes that input has (M+2)*D dimensions, where D is the dimensionality
of the target data. The first M*D features are treated as means, the next
M features as variances and the last M features as mixture components of the GMM.
"""
def gmm_activation_fn(x):
D = T.shape(x)[1]/(M+2)
return T.concatenate([x[:,:M*D-1], T.exp(x[:,M*D:2*M*D-1]), T.nnet.sigmoid(x[:,2*M*D:])], axis=1)
def gmm_loss(M):
"""
GMM loss function.
Assumes that y_pred has (M+2)*D dimensions and y_true has D dimensions.
The first M*D features are treated as means, the next M features as
variances and the last M features as mixture components of the GMM.
"""
def loss(m, M, y_true, y_pred):
D = T.shape(y_true)[1]
return (y_pred[:,(D+1)*M+m]/y_pred[:,D*M+m]) \
* T.exp(-T.sum(T.sqr(y_pred[:,D*m:(m+1)*D] - y_true))/(2*y_pred[:,D*M+m]**2))
def gmm_loss_fn(y_true, y_pred):
seq = T.arange(M)
result, _ = theano.scan(fn=loss,
outputs_info=None,
sequences=seq,
non_sequences=[M, y_true, y_pred])
return result.sum()
return gmm_loss_fnNote that for both the activation and the loss function I return another function handle, which takes the correct arguments (as specified by the keras API). However, this doesn't work with the Lambda layer:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<string>", line 8, in <module>
File "build/bdist.linux-x86_64/egg/keras/models.py", line 359, in compile
File "build/bdist.linux-x86_64/egg/keras/layers/containers.py", line 74, in get_output
File "build/bdist.linux-x86_64/egg/keras/layers/core.py", line 1022, in get_output
TypeError: arg 5 (closure) must be tupleAre there better ways to do this?
yosipk
ylqfp
yunzhou
erlebach
Hi all,
I try to build a MDN-like model with keras. This basically adds a GMM on top of a network, which can be modeled using appropriate activation functions, i.e. for a D-dimensional output and M mixture components, we would apply:
I tried to model this using Graph but the only thing I could come up with is to split the (intermediate) output of the network into three dense layers with M*D, M and M outputs, respectively, and then apply the activations mentioned above. However, I don't want to learn separate weights, but rather apply the three activations to different parts of the output (since this is typically done if I understand it correctly). I had a look into the Lambda layer too, but then again it seems that indexing in Theano isn't straightforward as well.
Any ideas?