Some hint about loading the npy with correlated paramters

[gitathrun]

Through the study on OpenAI's model, here is some useful information for developers who wrote their own version of mlstm and try to import OpenAI's model paramters. In mlstm function in encoder.py, defines the tensors' name, this is the baseline.

1. Computation Graphic and tensor Under the name scope model, there are three sub name scope:

   • embedding
     • tensors: w
   • out
     • tensors: w, b
   • rnn
     • tensors: b, gh, gmb, gmx, gx, wh, wmh, wmx, wx The tensors are listed as follow:
       1. tensor_name: model/embedding/w
       2. tensor_name: model/out/b
       3. tensor_name: model/out/w
       4. tensor_name: model/rnn/b
       5. tensor_name: model/rnn/gh
       6. tensor_name: model/rnn/gmh
       7. tensor_name: model/rnn/gmx
       8. tensor_name: model/rnn/gx
       9. tensor_name: model/rnn/wh
       10. tensor_name: model/rnn/wmh
       11. tensor_name: model/rnn/wmx
       12. tensor_name: model/rnn/wx

2. Table for the correlation between tensor and .npy files For detailed information about each tensor and which .npy it is correlated, please check the table

Name	Correlated-tensor	Array Shape	npy file index	line of code
params[0]	embedding/w	(256,64)	0	embd, line 23
params[1]	rnn/wx	(64, 16384)	1	mlstm, line 47
params[2]	rnn/wh	(4096, 16384)	hstack 2-5	mlstm, line 48
params[3]	rnn/wmx	(64, 4096)	6	mlstm, line 49
params[4]	rnn/wmh	(4096, 4096)	7	mlstm, line 50
params[5]	rnn/b	(16384,)	8	mlstm, line 51
params[6]	rnn/gx	(16384,)	9	mlstm, line 53
params[7]	rnn/gh	(16384,)	10	mlstm, line 54
params[8]	rnn/gmx	(4096,)	11	mlstm, line 55
params[9]	rnn/gmh	(4096,)	12	mlstm, line 56
params[10]	out/w	(4096, 256)	13	fc, line 31
params[11]	out/b	(256,)	14	fc, line 38

Hopyfully this would help.