search

h2oai / h2ogpt

Private chat with local GPT with document, images, video, etc. 100% private, Apache 2.0. Supports oLLaMa, Mixtral, llama.cpp, and more. Demo: https://gpt.h2o.ai/ https://gpt-docs.h2o.ai/
http://h2o.ai
Apache License 2.0
11.39k stars 1.25k forks source link

LlaMa 65B + h2oai/openassistant_oasst1_h2ogpt_graded 1 epoch #130

Open arnocandel opened 1 year ago

arnocandel commented 1 year ago

(h2ollm) ubuntu@cloudvm:~/h2o-llm$ CUDA_VISIBLE_DEVICES=1 python finetune.py --base_model=decapoda-research/llama-65b-hf --llama_flash_attn=False --train_8bit=True --micro_batch_size=1 --run_id=3 --data_path=h2oai/openassistant_oasst1_h2ogpt_graded &> 3.log

arnocandel commented 1 year ago

https://huggingface.co/h2oai/h2ogpt-research-oasst1-512-65b

arnocandel commented 1 year ago

Eval: like for https://github.com/h2oai/h2ogpt/issues/216#issuecomment-1579573101