[Feature Request] pre-built Docker image support

EleutherAI / lm-evaluation-harness

A framework for few-shot evaluation of language models.

https://www.eleuther.ai

MIT License

5.96k stars 1.58k forks source link

Open zsaladin opened 3 months ago

zsaladin commented 3 months ago

Is there any plan to support pre-built docker image on dockerhub or github package?

I think It will be very helpful and become more accessible to a lot of people.

haileyschoelkopf commented 3 months ago

We'd be glad to support this! It's a great idea.

Won't be currently prioritizing it so if anyone is able to contribute this we'd welcome contributions. Else, will put it on the todo list.

VMinB12 commented 2 months ago

An official lm-evaluation-harness Docker image would be great. I just found that BigScience hosts the following Docker image: https://github.com/orgs/bigcode-project/packages/container/package/evaluation-harness

yaronr commented 2 months ago