Closed tomweingarten closed 4 years ago
The use_evonorm parameter no longer exists, so using PKM with the encoder-decoder fails.
File "/home/tom/.local/lib/python3.8/site-packages/routing_transformer/encoder_decoder.py", line 64, in __init__ enc = RoutingTransformerLM(**enc_kwargs) File "/home/tom/.local/lib/python3.8/site-packages/routing_transformer/routing_transformer.py", line 604, in __init__ self.routing_transformer = RoutingTransformer(dim, depth, max_seq_len, heads = heads, window_size = window_size, local_attn_window_size = local_attn_window_size, causal = causal, weight_tie = weight_tie, ff_dropout = ff_dropout, attn_dropout = attn_dropout, attn_layer_dropout = attn_layer_dropout, layer_dropout = layer_dropout, n_local_attn_heads = n_local_attn_heads, ff_glu = ff_glu, reversible = reversible, ff_chunks = ff_chunks, kmeans_ema_decay = kmeans_ema_decay, receives_context = receives_context, context_window_size = context_window_size, rel_pos_emb = rel_pos_emb, pkm_layers = pkm_layers, pkm_num_keys = pkm_num_keys, _register_kmeans_update = _register_kmeans_update) File "/home/tom/.local/lib/python3.8/site-packages/routing_transformer/routing_transformer.py", line 564, in __init__ fn_wrapper(get_parallel_fn()) File "/home/tom/.local/lib/python3.8/site-packages/routing_transformer/routing_transformer.py", line 551, in <lambda> get_pkm = lambda: PKM(dim, num_keys = pkm_num_keys, use_evonorm = False) TypeError: __init__() got an unexpected keyword argument 'use_evonorm'
https://github.com/lucidrains/routing-transformer/commit/0ad636aad6d3c16d1430ff18bba9e1447581cddc oops, fixed!
The use_evonorm parameter no longer exists, so using PKM with the encoder-decoder fails.