search

aws-neuron / transformers-neuronx

Apache License 2.0
95 stars 28 forks source link

from_pretrained is broken after transformers made safetensor serialization default #60

Closed dennj closed 10 months ago

dennj commented 10 months ago

https://github.com/aws-neuron/transformers-neuronx/blob/33fa412447a4028edb252fd06aae9ed93086a450/src/transformers_neuronx/module.py#L144C74-L144C74

The function is looking for pytorch_model.bin that does not exist.

Because of it most of the tutorials are broken, this one for example: https://github.com/aws-neuron/aws-neuron-samples/blob/master/torch-neuronx/transformers-neuronx/inference/meta-llama-2-13b-sampling.ipynb

jyang-aws commented 10 months ago

thanks @dennj This is the same issue as https://github.com/aws-neuron/aws-neuron-sdk/issues/786 we'll track it there.