johnsmith0031 / alpaca_lora_4bit

MIT License
534 stars 84 forks source link

Add Weights & Biases integration #99

Closed rakovskij-stanislav closed 1 year ago

rakovskij-stanislav commented 1 year ago

This PR adds wandb integration for alpaca_lora_4bit

wandb is an instrument that integrates in torch to make a good training process visualization:

image image

There are visualizations generated by wandb. It helps to:

Also I found that your code does not use eval dataset at all despite it prepares it: https://github.com/johnsmith0031/alpaca_lora_4bit/blob/main/train_data.py#L166 So every time 20% of dataset was just unused.

johnsmith0031 commented 1 year ago

Thanks for your contribution!