search

jina-ai / jerboa

LLM finetuning
Apache License 2.0
41 stars 4 forks source link

feat: add save full models #102

Closed azayz closed 1 year ago

azayz commented 1 year ago

Closes https://github.com/jina-ai/jerboa/issues/99

This PR takes in already saved adapter weights, and model name and saves the full pytorch model weights to upload it in hf models and activate inference API

related discussion: https://github.com/huggingface/peft/issues/636

sebastian-weisshaar commented 1 year ago

The issue seems to be a change to transformers that does not allow for .to(device) if we use quantization: https://github.com/huggingface/transformers/blob/66954ea25e342fd451c26ec1c295da0b8692086b/src/transformers/modeling_utils.py#L1897. Trying to fix this with the PR: https://github.com/jina-ai/jerboa/pull/104.