I think that when the model is initialized in line 39, a local_attn_window_size parameter is missing. If there is only n_local_attn_heads parameter but no local_attn_window_size parameter, an error will be reported.
In addition, I suggest that the author re-run other examples, it seems that there are still some problems. I believe this will make this job more perfect.
I think that when the model is initialized in line 39, a local_attn_window_size parameter is missing. If there is only n_local_attn_heads parameter but no local_attn_window_size parameter, an error will be reported. In addition, I suggest that the author re-run other examples, it seems that there are still some problems. I believe this will make this job more perfect.