Adds example with text generation web ui for local models

[paolorechia]

Tested with WizardLM 7B, did not work super well, but could be iterated on. I think combining this with the guidance library could be really nice BTW Not sure if you'd like to merge, but here goes :)

[yhyu13]

Thanks for contributing @paolorechia

Forgive my ignorance, what is guidance library?

Based on your changes, so the idea is to startup a textuwebui service, and then post request to the server? Would you like to make the server address be editable? If I get it correctly, it has been fixed at localhost:5000 in your code.

Why is WizardLM 7B fails? Did ist not follow the ToT prompt faithfully? I've tried AgentLLM which use local LLM instance to replace chatgpt, it also uses WizardLM7B by default, and WizardLM7B fails to follow the prompt (it does not respond to "say NOTHING ELSE" correctly and spit out extra info that I do not need). So I am not sure if it's the problem of the model it self.

[paolorechia]

Exactly, the model fails to follow the instructions as expected.

Guidance is a library which fits really nice into this use case: https://github.com/microsoft/guidance

[paolorechia]

Regarding the URL, I don’t have an use case where it needs to changed - but if you prefer we could parametrize it

[yhyu13]

Exactly, the model fails to follow the instructions as expected.

Guidance is a library which fits really nice into this use case: https://github.com/microsoft/guidance

I am going to follow this tutorial : https://github.com/microsoft/guidance/blob/main/notebooks/tutorial.ipynb. It seems it bit like LangChain's prompt template.

[marelle65187512]

Great job on the pull request, it's been merged

[paolorechia]

I think we can close this, as using the guidance version would make more sense, right? Thanks for the work here!