search

dilab-zju / self-speculative-decoding

Code associated with the paper **Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding**
Apache License 2.0
131 stars 8 forks source link

Can you share your prompt of LLaMA-2-70B? #3

Closed jaemin-han closed 11 months ago

jaemin-han commented 11 months ago

I would like to conduct an Autoregressive baseline experiment using the LLaMA-2-70B model on the CNN/DM dataset. However, I am unsure about the appropriate prompt to ensure accurate model inference. Could you kindly share the prompt you recommend for LLaMA-2-70B?

what I tried:

  1. summary: [text want to summarize]
  2. [text summarization]: [text text want to summarize]
  3. Write a concise summary of the text, return your responses with 5 lines that cover the key points of the text. [text want to summarize] SUMMARY:
  4. System: You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature. If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.
    [text want to summarize] summary:
junzhang-zj commented 11 months ago

We have updated the code and you can refer to it.

jaemin-han commented 11 months ago

Great! Thanks for the update! I'll take a look at the new code

2023년 10월 17일 (화) 오후 2:00, Jun Zhang @.***>님이 작성:

Closed #3 https://github.com/dilab-zju/self-speculative-decoding/issues/3 as completed.

— Reply to this email directly, view it on GitHub https://github.com/dilab-zju/self-speculative-decoding/issues/3#event-10673207855, or unsubscribe https://github.com/notifications/unsubscribe-auth/AR5WDV5IV6HQ7A2BLJDNP33X7YGINAVCNFSM6AAAAAA6BRS6W2VHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJQGY3TGMRQG44DKNI . You are receiving this because you authored the thread.Message ID: <dilab-zju/self-speculative-decoding/issue/3/issue_event/10673207855@ github.com>