jackabald / TiDB-Hack-NL-repo-search

Semantic Search Engine for Code Repositories

Apache License 2.0

4 stars 2 forks source link

Integrate Dify.ai #28

Closed jackabald closed 3 weeks ago

jackabald commented 1 month ago

https://docs.dify.ai/guides/model-configuration

afzal442 commented 3 weeks ago

Any update will be great. @jackabald

afzal442 commented 3 weeks ago

https://huggingface.co/models?pipeline_tag=text-generation

Load model directly

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3.1-8B") model = AutoModelForCausalLM.from_pretrained("meta-llama/Meta-Llama-3.1-8B")

afzal442 commented 3 weeks ago

Any lookup here? https://www.npi.ai/docs

jackabald commented 3 weeks ago

So my research with Dify has kind of led me astray. I don't know if there is any implementation we could use specifically, I think it is more of a way to develop LLMs and RAGs in a low code way using agents. I think dify is better used if we were making like a new project.

afzal442 commented 3 weeks ago

Thanks for that. plz validate this idea of having the use of tokenizer

from transformers import AutoTokenizer, AutoModelForCausalLM