Closed Innixma closed 11 months ago
Added benchmark_repo.py
to show a comparison of runtime. Note that for longer zeroshot simulations, the gap will increase and eventually be larger than the gap of load time. Regardless, both memmap and memopt are close in speed while being much faster than mem. I think it is good to support both, with memmap
being the default in most cases.
Add PredictionsOpt, fixed restrict_models bug, code cleanup
prediction_format
.