from transformers import AutoModelForCausalLM, AutoTokenizer
if __name__ == '__main__':
model_name = 'facebook/opt-30b'
model = AutoModelForCausalLM.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)
from parallelformers import parallelize
parallelize(model, num_gpus=8, fp16=True)
This error was thrown at parallelize method :
Bus error (core dumped)
We tried with parallelformers version 1.2.6 and transformers version 4.21.11, this error was not thrown. This error is only happening with the parallelformers version 1.2.7 and transformers version 4.21.11.
How to reproduce
This error was thrown at parallelize method :
We tried with parallelformers version 1.2.6 and transformers version 4.21.11, this error was not thrown. This error is only happening with the parallelformers version 1.2.7 and transformers version 4.21.11.
Environment