search

meta-llama / llama

Inference code for Llama models
Other
56.3k stars 9.56k forks source link

Fine-tunning or continue pre-train in other languages #483

Closed fernandosola closed 1 year ago

fernandosola commented 1 year ago

Llama2 is open for commercial use. However, the hugginface model card states that its use in other languages is out of scope (https://huggingface.co/meta-llama/Llama-2-7b and https://github.com/facebookresearch/llama/blob/main/MODEL_CARD.md). If we do a fine-tunning or continue the pre-training using datasets in pt-BR would we be infringing any license rules?

I work at Brasilian government and we intend continue the pre-train of llama2 with a lot more data in pt-Br to use it as baseline for several task specific future fine-tunnings. We want to open source this portuguese fluent baseline on huggingface. However we are concerned about licence issues. If you can not answer my question, can you suggest any comunication channel to meta so we can clear this doubt?

Chriss4123 commented 1 year ago

  1. License Rights and Redistribution.

    a. Grant of Rights. You are granted a non-exclusive, worldwide, non- transferable and royalty-free limited license under Meta's intellectual property or other rights owned by Meta embodied in the Llama Materials to use, reproduce, distribute, copy, create derivative works of, and make modifications to the Llama Materials.

    b. Redistribution and Use.

    i. If you distribute or make the Llama Materials, or any derivative works thereof, available to a third party, you shall provide a copy of this Agreement to such third party. ii. If you receive Llama Materials, or any derivative works thereof, from a Licensee as part of an integrated end user product, then Section 2 of this Agreement will not apply to you.

    iii. You must retain in all copies of the Llama Materials that you distribute the following attribution notice within a "Notice" text file distributed as a part of such copies: "Llama 2 is licensed under the LLAMA 2 Community License, Copyright (c) Meta Platforms, Inc. All Rights Reserved."

    iv. Your use of the Llama Materials must comply with applicable laws and regulations (including trade compliance laws and regulations) and adhere to the Acceptable Use Policy for the Llama Materials (available at https://ai.meta.com/llama/use-policy), which is hereby incorporated by reference into this Agreement.

    v. You will not use the Llama Materials or any output or results of the Llama Materials to improve any other large language model (excluding Llama 2 or derivative works thereof).

hbj52 commented 1 year ago

@fernandosola @Chriss4123

Hi, I have read the License Rights and Redistribution. But still I am a little bit confused of the situation of applying llama2 into commercial use in other languages, because of the out-of-scope uses in https://huggingface.co/meta-llama/Llama-2-70b.

Is it possible i get a simple reply from anyone of you?

Like, is it ok if I fine-tune llama2 as a model A and commercially utilized model A within a chatbot in other languages? Assuming that I have already send application to Meta AI and have received Get started with Llama 2 from Meta AI.

jspisak commented 1 year ago

Yes, there are a number of community projects fine tuning on different languages. Please go ahead and add support as long as you adhere to the license. Thanks!!

enpassanty commented 1 year ago

@fernandosola would love to see your approach if you choose to publish. I'm looking too further pre-train on custom data as well. Thanks

cesargm2015 commented 10 months ago

Hi @fernandosola, any updates on the fine tuned version in pt-br? It seems the Starling-LM weights are relatively good in pt-br, I’d continue from their checkpoint.

I’d appreciate if could share the progress

fernandosola commented 7 months ago

Our publish about continued pre-training for portuguese.

https://github.com/fernandosola/llm2govbr-publlm-aaai24