finetune error: Only Tensors of floating point and complex dtype can require gradients

[hanhanpp]

I try to full parameter finetune the llama-3 8b but meet errors:

File "/usr/local/lib/python3.10/dist-packages/bitorch_engine/layers/qlinear/nbit/layer.py", line 375, in initialize self.qweight = MPQWeightParameter( File "/usr/local/lib/python3.10/dist-packages/bitorch_engine/layers/qlinear/nbit/layer.py", line 51, in new return super().new(cls, data, requires_grad=requires_grad) File "/usr/local/lib/python3.10/dist-packages/torch/nn/parameter.py", line 40, in new return torch.Tensor._make_subclass(cls, data, requires_grad) RuntimeError: Only Tensors of floating point and complex dtype can require gradients

The qweight is INT32, but the torch requires floating point or complex dtype for requairing gradients.

[yanghaojin]

Hi hanhanpp,

Native pytorch currently does not support calculating gradients for INT32 tensors, so you need to install the adapted pytorch version we prepared. For specific installation instructions, please see here: https://github.com/GreenBitAI/bitorch-engine?tab=readme-ov-file#installation

BR Haojin