Hi, using xformers monkey patch training llama2 got loss explosion

[lucasjinreal]

By using xformers to train llama2, the loss are explosion, do u know why? On V100 only