Fix dataset - Githubissues

RUCAIBox / LLMBox

A comprehensive library for implementing LLMs, including a unified training pipeline and comprehensive model evaluation.

MIT License

578 stars 74 forks source link

Closed huyiwen closed 5 months ago

huyiwen commented 5 months ago

Fix log_predictions when batch > 1
Fix multiple datasets evaluation by setting dataset arguments separately for each dataset
Support specifying model_backend to evaluate models not included in enumerations
Remove redundant spaces of target
Reload tokenizer to enable tokenizer_parallelism
Prepare for huggingface prefix_caching