search

petals-infra / chat.petals.dev

💬 Chatbot web app + HTTP and Websocket endpoints for LLM inference with the Petals client
https://chat.petals.dev
308 stars 79 forks source link

Change separators for BLOOMZ in the few-shot mode #14

Closed borzunov closed 1 year ago

borzunov commented 1 year ago

Currently, BLOOMZ behaves well only for the first output in the few-shot mode, then outputs </s> and forgets everything. This is visible in the English-to-Spanish translation example.

We need to use stop_sequence = "\n\n" and extra_stop_sequences = ["</s>"] to fix this.

DarkIlluminatus commented 1 year ago

good idea.

borzunov commented 1 year ago

Few-shot mode was removed, since we decided to focus on supporting multiple model families instead.