LLama2-70B

[Pefect96]

由于网络以及权限问题，我们无法在reasoning_and_editing.py调用GPT-3.5-Turbo，请问能否提供使用LLama2来进行标题编辑生成的代码呢？

[yzy-bupt]

You can easily run the Llama 2 code from the official repository at https://github.com/meta-llama/llama/tree/main and then modify the inference code. For example, you can use https://github.com/meta-llama/llama-recipes/blob/main/recipes/quickstart/inference/local_inference/inference.py to debug prompts, and then use https://github.com/meta-llama/llama/blob/main/example_chat_completion.py for batch inference with loops. You can directly use the prompt we provided in https://github.com/yzy-bupt/LDRE/blob/main/src/reasoning_and_editing.py to reproduce our results.

Additionally, our experiments have shown that Llama's performance is not as good as expected. We recommend using OpenAI's GPT series instead. The GPT3.5-turbo we used is already outdated, and GPT-4/4o might offer even better performance and potentially yield better results.

Finally, we welcome you to follow our repository. We will be releasing the JSON files for GPT3.5-turbo inference soon.

[Pefect96]

Thanks for your reply. What type of GPU do I need to use if I want to use the llama-70B for inference? Finially, I will follow this remarkable repository, and i am waiting for the JSON files edieted by the GPT3.5-turbo.

[yzy-bupt]

We are using 8 A100 80G GPUs. We still recommend using OpenAI's GPT for generation. If you are experiencing network issues (e.g., your server does not have VPN access), we suggest using Google Colab (https://colab.research.google.com/). It allows you to access OpenAI and is very fast!

[Pefect96]

Thanks for your reply. Acutally, using the openai api is too expensive. Thus, I am waiting for the JSON files edieted by the GPT3.5-turbo.