search

Lightning-AI / lit-llama

Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.
Apache License 2.0
5.89k stars 507 forks source link

Any luck finetuning? #131

Closed ardywibowo closed 1 year ago

ardywibowo commented 1 year ago

I'm currently running finetune_lora.py to test reproducibility on the Alpaca dataset. Currently at around iteration 13,000 and it has not output reasonable responses to the validation prompt yet. How long do you typically finetune until you start seeing generations that make sense?

Some details of my setup:

Thanks!

lantiga commented 1 year ago

Take a look at this improved branch for LLaMA-Adapter: https://github.com/Lightning-AI/lit-llama/pull/128, this produces very high-quality generations and it converges quickly. We'll be merging this in the next few hours.

We'll be looking at optimizing LoRA next.

ardywibowo commented 1 year ago

Thanks! I will check it out.

Any tips on using 16-mixed vs. bfloat16 (my GPU doesn't support bfloat16)? I think it shouldn't make much of a difference, but any anecdotal data points of figuring out how to set hyperparameters to finetune this would be very useful :D

lantiga commented 1 year ago

128 has now landed, you should be able to just change dtype to float16 in the finetune_adapter.py script without major issues I believe.

SHENZHENYI commented 1 year ago

I have run 'finetune_lora.py' for 7,5000 iterations when the micro-batch size is 2. (lora-r=16)

Don't know if your issue comes from #125.

And when you do generation. The current generate function doesn't stop when it hits . I fixed it myself.

Below are the prompted results. I think they are fairly good.

'''

Instruction:

Tell me about alpacas.

Response:

Alpacas are a type of domesticated animal that originated in the high altitude areas of South America. They belong to the camelid family, and are related to camels, llamas, and vicuñas. They are relatively small and stocky, with shaggy, long coats. Their coat comes in two varieties: suri and huacaya. Suris have long, silky, and curly coats, while huacayas have coarser, flat, and dense coats. Alpacas are bred for their meat, wool, and hide, and are raised for their fiber. They are known for their docile temperament and are often considered domesticated animals. They are typically kept on small farms or ranches in their natural habitat.

Instruction:

Tell me about the president of Mexico in 2019.

Response:

Most of the information you may wish to know about President Andrés Manuel López Obrador, commonly known as Andrés Manuel or AMLO, is already available in many online sources.

A brief summary of the current president of Mexico is as follows:

Instruction:

Tell me about the king of France in 2019.

Response:

The current king of France is Emmanuel Macron. He was elected in 2017 and was sworn in as the President of France on May 14, 2017. King Macron has not been on the throne since 1793, when the French monarchy was abolished during the time of the French Revolution. Since then, France has had the office of President. The most recent King of France was Charles X.

Instruction:

List all Canadian provinces in alphabetical order.

Response:

The list of Canadian Provinces in alphabetical order is:

  1. Alberta
  2. British Columbia
  3. New Brunswick
  4. Newfoundland and Labrador
  5. Nova Scotia
  6. Ontario
  7. Prince Edward Island
  8. Quebec
  9. Saskatchewan
  10. Yukon. '''
ardywibowo commented 1 year ago

Thanks! I ran the new code and it seems to be stable.

For others who don’t have a bfloat16 GPU, I ran ‘finetune_adapter.py’ with some parameters float and some params half-precision, and needed to use Deepspeed Stage 3 to get it to fit on my GPU.