search

nlpxucan / WizardLM

LLMs build upon Evol Insturct: WizardLM, WizardCoder, WizardMath
9.21k stars 713 forks source link

Model does not obey follow up instructions very well(at all), compared to Vicuna for example #7

Open dogjamboree opened 1 year ago

dogjamboree commented 1 year ago

I use these LLM's for creative writing and with Vicuna I'm able to get it to revise sections, like add or remove details, make things longer or shorter, etc (with varying degrees of success of course).

With WizardLM it sometimes give me a prompt saying, sure I'll do such and such, but it never changes anything.

victorsungo commented 1 year ago

Thanks for your kindly feedback.

We have also recently identified some shortcomings of WizardLM in part of 29 skills, including the academic writing you mentioned, and the coding issues feedback from others, and some reasoning problems, and so on. We have recorded all of your suggestions and taken them seriously.

We are committed to optimizing the Evol-Instruct algorithms now, and hope that these skills can be improved in the next version of the model.

We also hope that you can continue to provide us with more feedback. Thanks again. :heart:

CarlKenner commented 1 year ago

WizardLM has no concept of follow-up instructions. There's no tag to indicate the start of a question. Its input is formatted like:

If a car travels 120 miles in 2 hours, what is its average speed in miles per hour?

### Response:

It's not at all clear how you would even send it a follow-up question.

What are you using to run the model?

dogjamboree commented 1 year ago

I use either llama.cpp or oobabooga with versions of the model freely available on HuggingFace.

teknium1 commented 1 year ago

WizardLM has no concept of follow-up instructions. There's no tag to indicate the start of a question. Its input is formatted like:

If a car travels 120 miles in 2 hours, what is its average speed in miles per hour?

### Response:

It's not at all clear how you would even send it a follow-up question.

What are you using to run the model?

You would send a followup by placing the previous conversation into the first section and then your followup after that, and then place a response tag

CarlKenner commented 1 year ago

I use either llama.cpp or oobabooga with versions of the model freely available on HuggingFace.

Well, there's your problem. Oobabooga doesn't have WizardLM support yet. (Although I have a pull request which adds it.)

dogjamboree commented 1 year ago

Llama.cpp supports it... I mean it responds with answers that tell me it understood my request such as saying " I'm sorry for my previous request, here's what you requested... " or something similar. So something is happening behind the scenes.

On Fri, Apr 28, 2023, 6:27 AM Carl Kenner @.***> wrote:

I use either llama.cpp or oobabooga with versions of the model freely available on HuggingFace.

Well, there's your problem. Oobabooga doesn't have WizardLM support yet. (Although I have a pull request which adds it.)

— Reply to this email directly, view it on GitHub https://github.com/nlpxucan/WizardLM/issues/7#issuecomment-1527563852, or unsubscribe https://github.com/notifications/unsubscribe-auth/A4WD5AHWDCNZBG7FRSIPVI3XDPATNANCNFSM6AAAAAAXNCO5L4 . You are receiving this because you authored the thread.Message ID: @.***>

xdevfaheem commented 1 year ago

WizardLM has no concept of follow-up instructions. There's no tag to indicate the start of a question. Its input is formatted like:

If a car travels 120 miles in 2 hours, what is its average speed in miles per hour?

### Response:

It's not at all clear how you would even send it a follow-up question.

What are you using to run the model?

use langchain lammacpp