Open yeliu918 opened 3 weeks ago
Hi @archersama,
We have a family of new embedding models, CodeXEmbed, and would like to submit its results to our leaderboard. Could you please assist us with this?
{"Model": "CodeXEmbed-7B", "Model Size (Million Parameters)": 7B, "URL": "”, "Apps": 85.22, "CosQA": 33.27, "Synthetic Text2sql":64.57, "CodeSearchNet": 78.84, "CodeSearchNet-CCR": 86.77, "CodeTrans-Contest": 90.64, "CodeTrans-DL": 32.31, "StackOverFlow QA": 94.25, "CodeFeedBack-ST": 80.93, "CodeFeedBack-MT": 57.83, "Avg": 70.46}
{"Model": "CodeXEmbed-2B", "Model Size (Million Parameters)": 2B, "URL": "”, "Apps": 74.99, "CosQA": 36.31, "Synthetic Text2sql":59.00, "CodeSearchNet": 73.50, "CodeSearchNet-CCR": 85.77, "CodeTrans-Contest": 86.63, "CodeTrans-DL": 33.17, "StackOverFlow QA": 90.54, "CodeFeedBack-ST": 81.15, "CodeFeedBack-MT": 53.08, "Avg": 67.41}
{"Model": "CodeXEmbed-400M", "Model Size (Million Parameters)": 2B, "URL": "”, "Apps": 48.57, "CosQA": 34.05, "Synthetic Text2sql":58.96, "CodeSearchNet": 72.53, "CodeSearchNet-CCR": 80.15, "CodeTrans-Contest": 75.67, "CodeTrans-DL": 34.85, "StackOverFlow QA": 89.51, "CodeFeedBack-ST": 78.87, "CodeFeedBack-MT": 45.75, "Avg": 61.89}
Great Work!!! But could you provide some URLs or API endpoints? This would help prevent individuals from submitting unverifiable models, which could potentially compromise the integrity of the leaderboard.
Hi @archersama,
We have a family of new embedding models, CodeXEmbed, and would like to submit its results to our leaderboard. Could you please assist us with this?
{"Model": "CodeXEmbed-7B", "Model Size (Million Parameters)": 7B, "URL": "”, "Apps": 85.22, "CosQA": 33.27, "Synthetic Text2sql":64.57, "CodeSearchNet": 78.84, "CodeSearchNet-CCR": 86.77, "CodeTrans-Contest": 90.64, "CodeTrans-DL": 32.31, "StackOverFlow QA": 94.25, "CodeFeedBack-ST": 80.93, "CodeFeedBack-MT": 57.83, "Avg": 70.46}
{"Model": "CodeXEmbed-2B", "Model Size (Million Parameters)": 2B, "URL": "”, "Apps": 74.99, "CosQA": 36.31, "Synthetic Text2sql":59.00, "CodeSearchNet": 73.50, "CodeSearchNet-CCR": 85.77, "CodeTrans-Contest": 86.63, "CodeTrans-DL": 33.17, "StackOverFlow QA": 90.54, "CodeFeedBack-ST": 81.15, "CodeFeedBack-MT": 53.08, "Avg": 67.41}
{"Model": "CodeXEmbed-400M", "Model Size (Million Parameters)": 2B, "URL": "”, "Apps": 48.57, "CosQA": 34.05, "Synthetic Text2sql":58.96, "CodeSearchNet": 72.53, "CodeSearchNet-CCR": 80.15, "CodeTrans-Contest": 75.67, "CodeTrans-DL": 34.85, "StackOverFlow QA": 89.51, "CodeFeedBack-ST": 78.87, "CodeFeedBack-MT": 45.75, "Avg": 61.89}