Support of Calling LLM (cloud GPU) - Githubissues

michaelthwan / searchGPT

Grounded search engine (i.e. with source reference) based on LLM / ChatGPT / OpenAI API. It supports web search, file content search etc.

MIT License

621 stars 65 forks source link

Support of Calling LLM (cloud GPU) #10

Closed michaelthwan closed 1 year ago

michaelthwan commented 1 year ago

Allow the option for users to not use OpenAI API (which need API key)

[x] Fine-tuned GPT-NeoX 20B (@michaelthwan) blocked, too expensive
[x] GPT-NeoX 20B (from Goose AI) (@michaelthwan )
[x] Bloom xxB (created another card)

michaelthwan commented 1 year ago

https://nlpcloud.com/#plans On CPU: $0.003 per request On GPU: $0.005 per request GPT-J: + $0.00001 per token Fast GPT-J: + $0.00003 per token GPT-NeoX 20B: + $0.00004 per token Fine-tuned GPT-NeoX 20B: + $0.00007 per token Stable Diffusion: + $0.05 per generated image Whisper: + $0.0006 per second (duration of your audio or video file)

michaelthwan commented 1 year ago

https://goose.ai/ GPT-Neo 1.3B, Fairseq 1.3B Small $0.000110 /request GPT-J 6B, Fairseq 6B Medium $0.000450 /request Fairseq 13B Large $0.001250 /request GPT-NeoX 20B Massive $0.002650 /request

michaelthwan commented 1 year ago

Fine-tuned GPT-NeoX 20B: + $0.00007 per token -> 0.07 per 1k token

Davicci: 0.02 per 1k token

michaelthwan commented 1 year ago

done GPT-NeoX 20B (from Goose AI)

michaelthwan commented 1 year ago

Rename this task as call API and close