llmware-ai / llmware

Unified framework for building enterprise RAG pipelines with small, specialized models
https://llmware-ai.github.io/llmware/
Apache License 2.0
6.58k stars 1.52k forks source link

"Error binding parameter 22: type 'list' is not supported" when adding website using sqlite engine #1042

Open AndhikaWB opened 1 month ago

AndhikaWB commented 1 month ago

My requirements.txt:

streamlit==1.39.0
llmware==0.3.7
# Progress bar
ipywidgets==8.1.5
# Web scraping
beautifulsoup4==4.12.3
lxml==5.3.0

My sample code:

from llmware.configs import LLMWareConfig
from llmware.models import ModelCatalog
from llmware.library import Library

# Set engine to save library data
LLMWareConfig().set_active_db('sqlite')

lib = Library().create_new_library(
    account_name = 'dhika',
    library_name = 'test_lib'
)

urls = [
    'https://en.tempo.co/read/1915490/prabowo-indifferent-to-gibrans-alleged-fufufafa-account-says-dasco',
    # More URLs after that
]

print(len(urls))

for url in urls:
    lib.add_website(
        url = url,
        # Don't search site recursively
        get_links = False
    )

Error result:

image

image

AndhikaWB commented 1 month ago

After a bit debugging, the error is caused by user_tags which is a list, but not supported by sqlite

https://github.com/llmware-ai/llmware/blob/b8c1f0911f1e82377c57d49d60658a5c2a39c0db/llmware/parsers.py#L3038

image