Removes some altpredictor stuff and adds in 2-dimensional positional encoding instead of just doing positional encoding on the sequence dimension. Seems to help a little and doesn't cost anything.
This also adds a bit of logic to shorten the input sequence when running 'evalbam', so we can experiment a little and see if it helps anything to have shorter input sequences.
Removes some altpredictor stuff and adds in 2-dimensional positional encoding instead of just doing positional encoding on the sequence dimension. Seems to help a little and doesn't cost anything. This also adds a bit of logic to shorten the input sequence when running 'evalbam', so we can experiment a little and see if it helps anything to have shorter input sequences.