yiyepiaoling0715 commented 7 months ago

as the title, wheter other language as Java could use the base llm?

ChiYeungLaw commented 7 months ago

Other language can be used too.

CrisRodriguez commented 3 months ago

Hi @zheng5yu9, I post this so anyone having the same doubt can easily find an answer :)

WizardCoder-Python-34B-V1.0 is based on Code-lama 34B python

It is a non-instruct model
Data that the model saw :
- Same as llama-2 foundation model
- 500B tokens from a code-heavy dataset
- 100B tokens using a Python-heavy dataset Image taken from code-llama paper
It covers then a diverse set of programming languages. Among them: Python, C++, Java, PHP, TypeScript, C#, and Bash (with results reported in the paper)

WizardCoder-Python-34B-V1.0 is finetuned

Complex instruction fine-tuning, by adapting the Evol-Instruct method to the domain of code
Base dataset for evol-instruct: Code Alpaca
Code Alpaca covers also a diverse set of programming languages

So, even if WizardCoder-Python-34B-V1.0 is specialized in Python it covers still plenty of other programming languages!