Use LLMs other than GPT

[zohaibAsif-tes]

Model

• Falcon 7B Instruct

Steps

• [x] Use openllm library to load the model
• [x] Pass the model to langchain library
• [x] Test the code

Error

• Segmentation fault (core dumped) occurs when the process requires additional memory that the operating system does not permit access

Conclusion

• Use model quantization techniques
• https://github.com/zohaibAsif-tes/nl-to-sql/issues/3#issuecomment-1698693797

[zohaibAsif-tes]

Model Quantization

• [x] Use bitsandbytes library to load 4 bit model
• [x] Test the code

Error

• The model was successfully loaded
• The test resulted in this error: Invalid input. Keyword argument `table` should be either of type `dict` or `list`, but is <class 'str'>)

Conclusion

• Tried to find solutions for this error but could not do so
• Debugged the code and there was a garbage query generated
• Will implement this if a solution is found
• Going to follow this approach for now #2