search

defog-ai / sqlcoder

SoTA LLM for converting natural language questions to SQL queries
Apache License 2.0
3.27k stars 205 forks source link

defog/sqlcoder model size is big, can't we have API for this model #51

Closed range-yugen closed 6 months ago

range-yugen commented 9 months ago

First of all, thank you for this excellent GitHub repo. i have implemented it and am able to get good results. but here my one concern is that loading this model from hugging face is taking so much time (~5 minutes). do you have any API call for this model so that it will be easy to load it quickly? please let me know if you have as soon as possible.

NPap0 commented 9 months ago

First of all, thank you for this excellent GitHub repo. i have implemented it and am able to get good results. but here my one concern is that loading this model from hugging face is taking so much time (~5 minutes). do you have any API call for this model so that it will be easy to load it quickly? please let me know if you have as soon as possible.

I'm in need of an api/ per token pricing service (just like openai) as well. The company I work for is under 50 and defog does not offer service for smaller companies currently. (Hosting the model 24/7 is too pricey for us). Maybe if we get many requests here then its possible for defog to host a model that will serve api calls.

github-actions[bot] commented 7 months ago

Stale issue message

github-actions[bot] commented 6 months ago

This issue is stale because it has been open for 30 days with no activity.

github-actions[bot] commented 6 months ago

This issue was closed because it has been inactive for 14 days since being marked as stale.