How to extract the cell state and hidden state from an RNN model in CNTK?

[noble6emc2]

I know there is a way for tensorflow to do this, but I could not find any instructions neither online or in API document. I'm new to CNTK and wonder how to do this.

[noble6emc2]

To be more specific, I only need the previous hidden state, because it is crucial in attention modeling. So is there way to do it? Best regards.

[CynsierWang]

It sounds like a delay layer.

[CynsierWang]

def OneWordLookback(): x = C.placeholder() apply_x = C.splice(x, C.sequence. past_value(x)) return apply_x

def create_model(): with C.layers.default_options(initial_state=0.1): return C.layers.Sequential([ C.layers.Embedding(emb_dim), OneWordLookback(), C.layers.Recurrence(C.layers.LSTM(hidden_dim), go_backwards=False), C.layers.Dense(num_labels) ])` past_value() function returns the past value. The resulting tensor has the same shape as the input but is the previous logical sample.

[ke1337]

Can you try layers.RecurrenceFrom?

[whuala]

@CynsierWang I noticed that there is a placeholder in OneWordLookback(). But how can you bound it to an actual variable? Does something do it automatically?