search

lucidrains / routing-transformer

Fully featured implementation of Routing Transformer
MIT License
284 stars 30 forks source link

use_evonorm no longer supported in PKM #3

Closed tomweingarten closed 4 years ago

tomweingarten commented 4 years ago

The use_evonorm parameter no longer exists, so using PKM with the encoder-decoder fails.

File "/home/tom/.local/lib/python3.8/site-packages/routing_transformer/encoder_decoder.py", line 64, in __init__
    enc = RoutingTransformerLM(**enc_kwargs)
  File "/home/tom/.local/lib/python3.8/site-packages/routing_transformer/routing_transformer.py", line 604, in __init__
    self.routing_transformer = RoutingTransformer(dim, depth, max_seq_len, heads = heads, window_size = window_size, local_attn_window_size = local_attn_window_size, causal = causal, weight_tie = weight_tie, ff_dropout = ff_dropout, attn_dropout = attn_dropout, attn_layer_dropout = attn_layer_dropout, layer_dropout = layer_dropout, n_local_attn_heads = n_local_attn_heads, ff_glu = ff_glu, reversible = reversible, ff_chunks = ff_chunks, kmeans_ema_decay = kmeans_ema_decay, receives_context = receives_context, context_window_size = context_window_size, rel_pos_emb = rel_pos_emb, pkm_layers = pkm_layers, pkm_num_keys = pkm_num_keys, _register_kmeans_update = _register_kmeans_update)
  File "/home/tom/.local/lib/python3.8/site-packages/routing_transformer/routing_transformer.py", line 564, in __init__
    fn_wrapper(get_parallel_fn())
  File "/home/tom/.local/lib/python3.8/site-packages/routing_transformer/routing_transformer.py", line 551, in <lambda>
    get_pkm = lambda: PKM(dim, num_keys = pkm_num_keys, use_evonorm = False)
TypeError: __init__() got an unexpected keyword argument 'use_evonorm'
lucidrains commented 4 years ago

https://github.com/lucidrains/routing-transformer/commit/0ad636aad6d3c16d1430ff18bba9e1447581cddc oops, fixed!