Open dail45 opened 11 months ago
1) using orjson instead of standart python json, another small changes for better performance 2) accumulating chunks from openai when js in decoding chunks (config->backend_config->join_chunks turn on by default)
1) using orjson instead of standart python json, another small changes for better performance 2) accumulating chunks from openai when js in decoding chunks (config->backend_config->join_chunks turn on by default)