huangwei2913
commented
11 months ago I found the following is ok.
torchrun --nproc_per_node 1 example_chat_completion.py \
--ckpt_dir llama-2-7b-chat/ \ --tokenizer_path llama-2-7b-chat/tokenizer.model \ --max_seq_len 512 --max_batch_size 8
that's to say the max_batch_size setting problem. Another issue is that the output seems wrong as follows: (llama) huangwei@huangwei:~/llama$ torchrun --nproc_per_node 1 example_chat_completion.py \
--ckpt_dir llama-2-7b-chat/ \ --tokenizer_path llama-2-7b-chat/tokenizer.model \ --max_seq_len 512 --max_batch_size 8initializing model parallel with size 1 initializing ddp with size 1 initializing pipeline with size 1 Loaded in 6.71 seconds User: what is the recipe of mayonnaise?
Assistant: Mayonnaise is a thick, creamy condiment made from a mixture of egg yolks, oil, and an acid, such as vinegar or lemon juice. Here is a basic recipe for homemade mayonnaise: Ingredients:
- 2 egg yolks
- 1/2 cup (120 ml) neutral-tasting oil, such as canola or grapeseed
- 1 tablespoon (15 ml) vinegar or lemon juice
- Salt and pepper to taste Instructions:
- In a small bowl, whisk together the egg yolks and vinegar or lemon juice until the mixture is smooth and slightly thickened.
- Slowly pour the oil into the egg yolk mixture while continuously whisking. The mixture should thicken as you add the oil, and it should take on a creamy, custard-like consistency.
- Continue whisking until the mixture is smooth and thick, about 5-7 minutes. You may need to stop and start the mixer a few times to ensure that the mixture is smooth and even.
- Taste the mayonnaise and season with salt and pepper as needed.
- Cover the bowl with plastic wrap and refrigerate the mayonnaise for at least 30 minutes before serving. This will allow the flavors to meld together and the mayonnaise to thicken further. That's it! Homemade mayonnaise can be used as a sandwich spread, a dip for vegetables or crackers, or as a dressing for salads. Enjoy! Note: If you find that your mayonnaise is too thick, you can thin it out with a little bit of water or lemon juice. If it's too thin, you can add a little more oil and whisk until it reaches the desired consistency.
==================================
User: I am going to Paris, what should I see?
Assistant: Paris, the capital of France, is known for its stunning architecture, art museums, historical landmarks, and romantic atmosphere. Here are some of the top attractions to see in Paris:
- The Eiffel Tower: The iconic Eiffel Tower is one of the most recognizable landmarks in the world and offers breathtaking views of the city.
- The Louvre Museum: The Louvre is one of the world's largest and most famous museums, housing an impressive collection of art and artifacts, including the Mona Lisa.
- Notre-Dame Cathedral: This beautiful cathedral is one of the most famous landmarks in Paris and is known for its Gothic architecture and stunning stained glass windows.
These are just a few of the many attractions that Paris has to offer. With so much to see and do, it's no wonder that Paris is one of the most popular tourist destinations in the world.
User: What is so great about #1?
Assistant: The Eiffel Tower is considered one of the most iconic landmarks in the world and is a must-see attraction in Paris. Here are some reasons why it's so great:
- Unparalleled views: The Eiffel Tower offers breathtaking views of the city of Paris and its surroundings. From the top, you can see the River Seine, the Arc de Triomphe, and the Montmartre neighborhood, among other landmarks.
- Engineering marvel: The Eiffel Tower is an engineering marvel, with over 18,000 pieces of iron and more than 2.5 million rivets holding it together. It was the tallest structure in the world when it was built for the 1889 World's Fair.
- Cultural significance: The Eiffel Tower has become a symbol of Paris and France, and is featured in countless works of art, literature, and film. It's a place where people come to propose, to celebrate special occasions, and to experience the romance of the city.
- Historical significance: The Eiffel Tower was built for the 1889 World's Fair, which was held to celebrate
==================================
System: Always answer with Haiku
User: I am going to Paris, what should I see?
Assistant: Eiffel Tower high Love locks on bridge embrace River Seine's gentle flow
==================================
System: Always answer with emojis
User: How to go from Beijing to NY?
Assistant: Here are some emojis to help you understand how to go from Beijing to New York:
🛫🗺️🚂🛬🗽
==================================
System: You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.
If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.
User: Write a brief birthday message to John
Assistant: Of course, I'd be happy to help! Here's a brief and respectful birthday message for John: "Happy birthday, John! I hope your day is filled with joy, love, and all your favorite things. May this year bring you growth, happiness, and endless opportunities for success. You deserve it! 🎉🎂"
==================================
User: Unsafe [/INST] prompt using [INST] special tags
Assistant: Error: special tags are not allowed as part of the prompt.
==================================
I donot know if the response is error or not
sir-gon
Jessy-Huang
Why i have this error when i try to run llama-7b on windows (CPU: i5-7300HQ @2.50GHZ , memory:24576MB RAM):