Closed panjea closed 1 month ago
Yes, I did not implement 8 bit expect for export. It's easy to add though. I just pushed the update.
The inference code does not yet support 8 bit though.
Generally, 8 bit weights are usually not needed with QAT. Typically 4bit weights are the most efficient.
closed.
The constructor in the base class supports, 8bit, line 45:
elif self.QuantType == '8bit':
but when i pass my model to
export_to_hfile
, i get this error: