Llama2

[xiezipeng-ML]

• libai sft model_0000499(base on Llama-2-7b-hf):

  {
  'results': {
      'hellaswag': {'acc': 0.5746863174666401, 'acc_stderr': 0.004933800927560528, 'acc_norm': 0.759609639514041, 'acc_norm_stderr': 0.004264472071282531}
  }, 
  'versions': {
      'hellaswag': 0
  }, 
  'config': {
      'model': 'llama', 'batch_size': 1, 'device': 'cuda:0', 'num_fewshot': 0, 'limit': None, 'bootstrap_iters': 100000
      }
  }

• Llama-2-7b-chat-hf

  hf-causal (pretrained=/data/home/xiezipeng/hf_models/meta-llama/Llama-2-7b-chat-hf), limit: None, provide_description: False, num_fewshot: 0, batch_size: None
  |  Task   |Version| Metric |Value |   |Stderr|
  |---------|------:|--------|-----:|---|-----:|
  |hellaswag|      0|acc     |0.5778|±  |0.0049|
  |         |       |acc_norm|0.7553|±  |0.0043|