search

Kiel-AI / gpt-neo-deployment

GPT-Neo Deployment using Docker and Opyrator with API & UI
MIT License
45 stars 10 forks source link

Hardware Requirements #4

Closed mitmelon closed 2 years ago

mitmelon commented 2 years ago

Please what are the hardware Requirements to use this?

Thanks.

mattinannt commented 2 years ago

@mitmelon You could theoretically run it on any machine you like as long as you have enough storage and memory to start the server (the big model is around 10GB). You can run it on a CPU only but it's very slow and takes minutes for a prediction. It's much faster on a GPU. I personally used an AWS p2 instance with a GPU to run the model & server.