search

ByungKwanLee / MoAI

[ECCV 2024] Official PyTorch implementation code for realizing the technical part of Mixture of All Intelligence (MoAI) to improve performance of numerous zero-shot vision language tasks.
MIT License
311 stars 32 forks source link

How to save a model with some uint8 weights to fp32? #21

Open cassiaaaaaa opened 5 months ago

cassiaaaaaa commented 5 months ago

Dear author, I have a question about the datatype saving. In your released model, the weights are fp32, for example "Model.layers.7.feed_forward.w1.Plora_A.weight", data type is torch.float32, and the shape is ([256, 4096]). After it is loaded by BitsAndBytesConfig in 4bit, it becomes torch.uint8 with shape ([524288, 1]). And I think it is trained on the latter format. The question is , after training, how did you recover the model weights into fp32 with original shape?

ByungKwanLee commented 5 months ago

I just loaded the parameters in safetensor!

I did not use from_pretrained after training due to the issue you said

I did just manally loaded them

cassiaaaaaa commented 5 months ago

I just loaded the parameters in safetensor!

I did not use from_pretrained after training due to the issue you said

I did just manally loaded them

Thank for your nice reply!