Open jezekra1 opened 2 weeks ago
This breaks for llama-3-0 in BAM adapter, because maximum number of tokens is 2048 for this model in BAM: property 'max_new_tokens' must be <= 2048.
property 'max_new_tokens' must be <= 2048.
https://github.com/i-am-bee/bee-api/blob/b709b3ba139cfdadbbec9686d37cdf261f7f9cb4/src/runs/execution/factory.ts#L55
This is something that should be re-configured on the model, there is no reason for the limitation
This breaks for llama-3-0 in BAM adapter, because maximum number of tokens is 2048 for this model in BAM:
property 'max_new_tokens' must be <= 2048.https://github.com/i-am-bee/bee-api/blob/b709b3ba139cfdadbbec9686d37cdf261f7f9cb4/src/runs/execution/factory.ts#L55
This is something that should be re-configured on the model, there is no reason for the limitation