[Phi3] System role missing - Githubissues

guidance-ai / guidance

A guidance language for controlling large language models.

MIT License

18.13k stars 1.01k forks source link

[Phi3] System role missing #885

Closed riedgar-ms closed 3 weeks ago

riedgar-ms commented 3 weeks ago

The bug

Trying to use with system(): with lm set to a Phi3 model, I got an UnsupportedRoleException. We have copied the template supplied by Phi3: https://github.com/guidance-ai/guidance/blob/325b2c87531e91993af2a3b99957cfa7fe4b5fdf/guidance/chat.py#L165 This indeed does not mention system. However: https://github.com/microsoft/Phi-3CookBook/blob/main/md/02.QuickStart/Huggingface_QuickStart.md#2-learn-about-phi-3-prompt-template

To Reproduce

See above

System info (please complete the following information):

OS (e.g. Ubuntu, Windows 11, Mac OS, etc.): Windows, but this will be universal
Guidance Version (guidance.__version__): Working from synced fork

Harsha-Nori commented 3 weeks ago

The phi-3 cookbook is out of date, they initially started with a system role but removed it later because the model wasn't trained for it. Latest official recommendation is to not use system prompts.