search

kubeedge / ianvs

Distributed Synergy AI Benchmarking
https://ianvs.readthedocs.io
Apache License 2.0
115 stars 46 forks source link

add a proposal of Smart Coding benchmark suite #120

Closed safe-b closed 3 weeks ago

safe-b commented 4 months ago

Smart Coding benchmark suite: built on KubeEdge-lanvs

What type of PR is this? /kind test /kind design

What this PR does / why we need it: The PR is a proposal to enhance a dataset by annotating the core code of the classic Python project, and then add project problems and App projects, etc., to finally complete the creation of the dataset and conduct test evaluation. Which issue(s) this PR fixes: #98 Fixes #

hsj576 commented 4 months ago

The PR should further elaborate on how to implement this feature in Ianvs.

safe-b commented 2 months ago

The logic of the subject is basically consistent with Meng Zhuo's government affairs model. The comment data set of the code model is basically the same as that of the government model. The difference lies in the content of the issue data set. One question may have multiple answers. The next step is training and testing.

MooreZheng commented 2 months ago

/lgtm

MooreZheng commented 3 weeks ago

/lgtm

MooreZheng commented 3 weeks ago

/approve

kubeedge-bot commented 3 weeks ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hsj576, MooreZheng

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/kubeedge/ianvs/blob/main/OWNERS)~~ [MooreZheng] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment