truncated_normal doesn't work with dense in cntk 2.2

[delzac]

Hi,

input_tensor = C.input_variable(10) model = Dense(100, init=C.truncated_normal(0.075, seed=7), name='layer1')(input_tensor)

this give me an error. Only truncated normal have this issue, other initialiser is fine.

File "C:\Anaconda3\envs\Cobra\lib\site-packages\cntk\layers\layers.py", line 138, in Dense init_weights = _initializer_for(init, Record(output_rank=output_rank)) File "C:\Anaconda3\envs\Cobra\lib\site-packages\cntk\layers\blocks.py", line 51, in _initializer_for init = initializer_with_rank(init, **rank_params) File "C:\Anaconda3\envs\Cobra\lib\site-packages\cntk\initializer.py", line 184, in initializer_with_rank return cntk_py.random_initializer_with_rank(initializer, output_rank, filter_rank) RuntimeError: invalid unordered_map<K, T> key

Did i do something wrong or is this a bug?

[ke1337]

This is a known issue with Dense. Before it's fixed, can you try following instead?

in_dim = 10
out_dim = 100
W = C.parameter((in_dim, out_dim), init=C.truncated_normal(0.075, seed=7))
b = C.parameter((out_dim), init=0)
x = C.input_variable(in_dim)
y = C.times(x, W) + b

[delzac]

Roger thanks!!

[jerusalemc]

@KeDengMS I have the same question when i use Convolution2D with the truncated_normal intializer, what can i do? thanks in advance!

[ke1337]

@jerusalemc You may try create the initial value using numpy

[jerusalemc]

@KeDengMS Thanks so much! I did as follows, and it works! import scipy.stats as stats X = stats.truncnorm(-0.2, 0.2, loc=0, scale=1) a = X.rvs((10,10,5)) conv = C.layers.Convolution2D(filter_shape=(10, 5), num_filters=10, activation=C.relu, init=a, init_bias= C.truncated_normal(0.1), reduction_rank=0)