Langchain integration

[Kshitizkhandel]

Is it possible to integrate sqlcoder with langchain?

I tried a couple of approaches:

Approach1: llm = HuggingFaceHub(repo_id="defog/sqlcoder-7b-2",huggingfacehub_api_token="", model_kwargs={"temperature":0, "max_length": 512}) db_chain = SQLDatabaseChain.from_llm(llm, db, verbose=True) db_chain.run('question')

Approach2: tokenizer, model = get_tokenizer_model("defog/sqlcoder-7b-2") prompt = generate_prompt(question, prompt_file, metadata_file) eos_token_id = tokenizer.eos_token_id pipe = pipeline( "text-generation", model=model, tokenizer=tokenizer, max_new_tokens=300, do_sample=False, return_full_text=False, # added return_full_text parameter to prevent splitting issues with prompt num_beams=5, # do beam search with 5 beams for high quality results ) hf_pipe = HuggingFacePipeline(pipeline=pipe) db_chain = SQLDatabaseChain.from_llm(hf_pipe, db=db, prompt=PROMPT, verbose=True) db_chain.run('question')

Apporach1 didn't work because the model is not loaded on the hf inference endpoint and would require a pro subscription Approach2 didn't work because of an internal hf tokenizer error.

Can you help suggest a solution that integrates langchain with sqlcoder?

[naimehao]

may be u can try ollama?

[4nte]

Ollama works fine with langchain

[naimehao]

Ollama works fine with langchain

However, ollama does not provide the sqlcoder-7b-2 model. I deployed and ran this model manually on the cloud computing platform, but there are still problems, such as GPU memory overflow.

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity.

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale.

[LozanoJohan]

rt