search

microsoft / aici

AICI: Prompts as (Wasm) Programs
MIT License
1.87k stars 76 forks source link

better error on prompt len exceeded in rllm #81

Open mmoskal opened 3 months ago

mmoskal commented 3 months ago

The following gives stop_reason: "failed" but no indication why.

import pyaici.server as aici

async def main():
    await aici.FixedTokens("Some long text that will serve as a prompt for the model to generate more text. This is a test of the AICI server." * 500)
    await aici.gen_text(max_tokens=250, store_var=f"function", stop_at="```")

aici.start(main())