search

meta-llama / llama-recipes

Scripts for fine-tuning Meta Llama with composable FSDP & PEFT methods to cover single/multi-node GPUs. Supports default & custom datasets for applications such as summarization and Q&A. Supporting a number of candid inference solutions such as HF TGI, VLLM for local or cloud deployment. Demo apps to showcase Meta Llama for WhatsApp & Messenger.
14.69k stars 2.14k forks source link

Llama 3.1 Code Interpreter file reference #610

Open jonatananselmo opened 3 months ago

jonatananselmo commented 3 months ago

I have searched in “Model Cards and Prompt Formats” and in the Github repositories about what is the correct way to reference a file in the prompt to be used by the code interpreter but I have not found an example. In the paper, they only mention a few examples where they do it in the following way: file_path = “path/to/file”. But I'm not sure if this is the right way as sometimes it doesn't work and it gets worse when the language is not English. In the repository “llama-agentic-system” there are some examples about using files, but the code has too many layers and I couldn't get to the prompt that is sent to the model. I double checked that the code interpreter tool was enabled by the line “Environment: ipython” at the system prompt. I also tried enabling the brave_search and wolfram_alpha tools, but the result is the same. Here is the prompt I'm trying:

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

Environment: ipython
Tools: brave_search, wolfram_alpha

Cutting Knowledge Date: December 2023
Today Date: 25 July 2024

You are a helpful assistant.<|eot_id|><|start_header_id|>user<|end_header_id|>

Give me a summary

file_path = "./document.docx"<|eot_id|><|start_header_id|>assistant<|end_header_id|>

Playing with the temperature parameter, sometimes the model responds with the <|python_tag|> token but sometimes not. I am using Llama 3.1 - 70B - fp8 by NeuralMagic

init27 commented 2 months ago

Hey @jonatananselmo thanks for the feedback and Q!

I will updating the repo with some example references on how to apply function calling along with tutorials, I will let you know once they are updated.

jonatananselmo commented 2 months ago

Great @init27, an example using a file reference should be helpful!

animeshj9 commented 1 month ago

Commenting to get notifications.

liu-zhy commented 1 month ago

@init27 Great! May I ask when it will be updated?

init27 commented 1 month ago

@liu-zhy Here is an example-let me know if you have any Qs!

liu-zhy commented 1 month ago

@init27 Thanks, but I still want to know how to operate on files when using Code Interpreter? For example, give a csv file, I want call Code Interpreter to visualize this csv file. How can I pass the file path to Code Interpreter?