search

Thytu / SMIT

SMIT: A Simple Modality Integration Tool
MIT License
15 stars 3 forks source link

Add a `classmethod` to load a trained model #3

Closed Thytu closed 5 months ago

Thytu commented 5 months ago

The current process of loading a trained model can be quite challenging for newcomers (cf), this can be both frustrating and discouraging for them.

SMIT should provide a classmethod that would automatically load a trained model given a path to a safe_tensor and the config file used to generate the model.

Ultimately it would great to not have to provide the config file by maybe storing everything into the safe_tensor file.

Thytu commented 5 months ago

It appears that Trainer currently does not provide a way to add metadata to the saved model.safetensors file.

A work-around would be to load the saved model.safetensors file, add the metadata, and re-save it again. It will add some processing time at the end of each save_steps but it will result in a more friendly way to load a train model.

A user will be able to simply provide a tensor tensor and SMIT could automatically adapt the model (LoRA / Quant) depending the config file provided.

As a fallback in case the safetensor file does not contain the config file in its metadata, SMIT should also provide a method that will takes as input both the safetensor and a config as dict/file.