Open mikestaub opened 3 months ago
As a user, I want to control the chat configuration dynamically.
Depending on the machine specs, the sliding_window_size and attention_sink_size may be needed to avoid a 'max window length exceeded' error for example.
sliding_window_size
attention_sink_size
Reference: https://github.com/mlc-ai/web-llm/blob/main/src/config.ts#L66
As a user, I want to control the chat configuration dynamically.
Depending on the machine specs, the
sliding_window_sizeandattention_sink_sizemay be needed to avoid a 'max window length exceeded' error for example.Reference: https://github.com/mlc-ai/web-llm/blob/main/src/config.ts#L66