search

bigcode-project / selfcodealign

[NeurIPS'24] SelfCodeAlign: Self-Alignment for Code Generation
https://arxiv.org/abs/2410.24198
Apache License 2.0
270 stars 18 forks source link

How does StarCoder-15b generates concept, instruction, and responses? #14

Open wasiahmad opened 2 weeks ago

wasiahmad commented 2 weeks ago

Is StarCoder-15b an instruct model itself? If not, how does it is utilized in concept, instruction, and response generation?

UniverseFly commented 2 weeks ago

Hi, StarCoder-15b is a base model, and we use it to generate data and create starcoder2-instruct. The main pipeline code is here: https://github.com/bigcode-project/selfcodealign/blob/46749cceeb84cf1773deb5c157b04930acebc5cd/src/star_align/self_ossinstruct.py.

wasiahmad commented 2 weeks ago

Thank you for getting back to me. I saw the code. Given a code snippet, isn't it hard to generate concepts/instructions for a base model? I saw the code and realized the prompt was quite simple and straight-forward. Did you evaluate how well the base model in generating concepts/instructions? Any analysis or findings from earlier experiments?

UniverseFly commented 2 weeks ago

You can find the instructions from an earlier selfcodealign, starcoder2-instruct. Sorry the instructions for the new version has not been updated yet. It's mostly compatible with the old one, but we will release an updated README soon.

Regarding the analysis, the section 4.1 (Self-Alignment with Different Models) in our paper might be of your interest.