RoboRate / TuningTitans

MIT License
0 stars 6 forks source link

feat: benchmark(5) Benchmark.py串接前端 #32

Closed FernandeChant closed 1 year ago

FernandeChant commented 1 year ago

目標

  1. 修改Benchmark.py
  2. 串app.py

    預計規劃

  3. 修改Benchmark.py: 原本只有產出我們自己的fine_tune_model報告,現在修改為fine_tune_model + raw model
  4. 串app.py:在app.py讀取jsonl檔案和fine_tune_model的名稱,並呼叫Benchmark.py的function

預期結果

app.py

`@app.route('/', methods=['GET', 'POST'])
 def BenchmarkReport():
     if request.method == 'POST':
         uploaded_file = request.files['jsonl_file']
         name = request.form['name']

         with NamedTemporaryFile(delete=False, suffix='.jsonl') as temp_file:
             temp_path = temp_file.name
             uploaded_file.save(temp_path)
         # Generate answers using the provided dataset_path and model_name
         dataset_path = temp_path`

Benchmark.py

`def generateAnswers(dataset_path, model_names):
    questions = []
    model_answers = {model_name: [] for model_name in model_names}
    ideal_answers = []

......
def evalWithRubric(test_set, model_names):
    rubric_evaluations = {model_name: [] for model_name in model_names}
    questions = test_set['Question']
    model_answers = test_set['ModelAnswers']
    ideal_answers = test_set['IdealAnswer']`
FernandeChant commented 1 year ago

已解決+PR