ShishirPatil gorilla issues

ShishirPatil / gorilla

Gorilla: Training and Evaluating LLMs for Function Calls (Tool Calls)

https://gorilla.cs.berkeley.edu/

Apache License 2.0

11.51k stars 1.01k forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

[bug] OpenFunctions-v2: <Issue>用bfcl测试我们微调的模型FunctionCall能力的实际效果

#791 gaochunrong opened 22 minutes ago
0
Add override and final headers to OSS handlers

#790 VishnuSuresh27 opened 1 hour ago
0
[BFCL Dataset Revamp 6/n] Live Relevance Data Fix

#789 Fanjia-Yan opened 4 hours ago
0
[BFCL] Non_Live Overall Accuracy、Non-Live AST Accuracy、AST Summary、Non-Live Exec Accuracy、Live Accurac这些指标数据如何评估某个模型的优劣区别？

#788 gaochunrong opened 4 hours ago
2
[BFCL] Add New Model `Qwen/Qwen2.5-72B-Instruct`

#787 HuanzhiMao closed 10 hours ago
0
[BFCL] some tiny fix in possible_answer

#786 zhangch-ss closed 1 day ago
0
[BFCL] The update frequency of the Berkeley Function Calling Leaderboard

#785 fengzhu1 opened 1 day ago
1
[bug] OpenFunctions-v2: <Issue>CUDA_VISIBLE_DEVICES=1 bfcl evaluate --model Qwen/Qwen2.5-7B-Instruct --test-category simple 执行这条命令，score目录生成的指标文件没有具体的数据。

#784 gaochunrong closed 7 hours ago
17
Add Weaviate APIs to Gorilla API Zoo

#783 CShorten opened 1 day ago
0
[BFCL Blog] Update Multi Turn Blog to Reflect New Metric from #733

#782 HuanzhiMao opened 2 days ago
0
[BFCL Chore] Add Retry Mechanism with Backoff for Rate Limit Handling Across Proprietary Models

#781 HuanzhiMao opened 2 days ago
0
[feature] Add CICD for BFCL

#780 CharlieJCJ opened 2 days ago
2
[BFCL Blog] Update BFCL V3 Blog Images

#779 HuanzhiMao opened 3 days ago
1
[BFCL] Dataset and Possible Answer Fix (Live Categories) for Illegal Python Parameter Name

#778 HuanzhiMao closed 2 days ago
1
[BFCL] Add Unit Test to Check for Illegal Python Parameter Name

#777 HuanzhiMao closed 2 days ago
3
[BFCL] javascript test category fails on checking Array argument's type

#776 leocnj closed 1 day ago
3
add transformers to pyproject.toml

#775 gabrielibagon closed 3 days ago
3
[BFCL] Error when consecutively generate from multiple oss_models with vllm

#774 YifanHao opened 4 days ago
1
[BFCL] Mismatch between local evaluation and leaderboard numbers

#773 sileix closed 3 days ago
6
[BFCL Dataset Revamp 5/n] Multi-Turn Base WrapUp

#772 Fanjia-Yan closed 2 days ago
2
[BFCL] Multi-turn query missing info & incorrect label

#771 mingzhu0527 opened 1 week ago
2
Fix handling of examples with no tools in Gemini

#770 gabrielibagon closed 1 week ago
0
Remove stop condition in gemini retry logic

#769 gabrielibagon closed 1 week ago
0
Skip adding empty content from gemini

#768 gabrielibagon closed 1 week ago
1
[BFCL] Remove Duplicate Line in `record_cost_latency`

#767 HuanzhiMao closed 1 week ago
0
[BFCL Website] Add Search Functionality for Leaderboard and Remove Multi-Turn Composite Column

#766 Sharonfish closed 2 days ago
2
[BFCL] Dataset Revamp Initiative

#765 Fanjia-Yan opened 1 week ago
0
Add exponential retry logic for gemini models

#764 gabrielibagon closed 1 week ago
0
[BFCL Dataset Revamp 4/n] Live Irrelevance

#763 Fanjia-Yan closed 3 days ago
1
[BFCL Chore] Supply `data_multi_turn.csv` for Multi-Turn Evaluation Results

#762 HuanzhiMao closed 1 week ago
0
[BFCL] Remove Workaround Patch for Vertex AI Package

#761 HuanzhiMao closed 1 week ago
0
Update google-cloud-aiplatform version to 1.72.0

#760 gabrielibagon closed 1 week ago
0
[BFCL] How to evaluate bitsandbytes 4bit and 8bit quantized models?

#759 abdul-456 opened 1 week ago
4
[RAFT] Add link to Azure RAFT Distillation Recipe

#758 cedricvidal closed 23 hours ago
0
[BFCL] How to add our own llm into Leaderboard?

#757 fengzhu1 closed 1 week ago
3
[BFCL] Introducing custom handlers

#756 sangmandu opened 1 week ago
1
[BFCL] Add support for Writer models and Palmyra X 004

#755 samjulien opened 1 week ago
3
[BFCL] Robustness Patch for `_multi_threaded_inference`

#754 HuanzhiMao closed 1 week ago
0
[BFCL] bugs in function def _multi_threaded_inference(self, test_case, include_input_log: bool, include_state_log: bool):

#753 pikepokenew closed 1 week ago
1
[BFCL] Update Llama Prompting Handler for Llama 3.0 vs. 3.1 Chat Template Differences

#752 HuanzhiMao closed 4 hours ago
0
[BFCL] Prompt Caching for Claude Models

#751 VishnuSuresh27 closed 1 week ago
0
[BFCL] Add `claude-3-5-haiku-20241022`, `claude-3-5-haiku-20241022-FC`, `claude-3-5-sonnet-20241022`, `claude-3-5-sonnet-20241022-FC`

#750 HuanzhiMao closed 6 days ago
0
[BFCL] Remove `Llama-3.2-3B-Instruct-FC` and `Llama-3.2-1B-Instruct-FC` from Leaderboard

#749 HuanzhiMao closed 1 week ago
0
[BFCL] Leaderboard Update, 11/17/2024

#748 HuanzhiMao closed 1 week ago
6
[BFCL] Minor Grammatical Corrections to DEFAULT_SYSTEM_PROMPT

#747 HuanzhiMao closed 1 week ago
0
[BFCL] Chore: Improve Inference Log Readability

#746 HuanzhiMao closed 2 weeks ago
0
[BFCL Dataset Revamp 4/n] Multi-Turn (Part 2)

#745 Fanjia-Yan closed 2 weeks ago
0
Updated readme

#744 YLu-1258 closed 2 weeks ago
0
[BFCL] Add Support for Regeneration, Specific Test Entry IDs, and Custom Directory Locations

#743 Raymond112514 closed 1 day ago
3
[BFCL] Added EventPlanningAPI for multi-turn for BFCL

#742 daanaea opened 2 weeks ago
0