search

nlpodyssey / cybertron

Cybertron: the home planet of the Transformers in Go
BSD 2-Clause "Simplified" License
283 stars 25 forks source link

Support loading safetensors format #43

Open gals opened 3 months ago

gals commented 3 months ago

By default classifiers look for pytorch_model.bin, which doesn't exist in newer versions of transformers:

pytorch_model.bin: no such file or directory

The trained model is stored using https://huggingface.co/docs/safetensors/en/index. For example, see https://github.com/huggingface/transformers/issues/28863.

matteo-grella commented 3 months ago

Hi @gals

We made some progresses here.

Do you have some capacity to integrate this safetensors implementation into cybertron?