search

OoriData / OgbujiPT

Client-side toolkit for using large language models, including where self-hosted
Apache License 2.0
101 stars 8 forks source link

Pgvector bigdata insert and search #57

Closed uogbuji closed 9 months ago

uogbuji commented 9 months ago

Changes for addressing #54 & #55.

Update: plus #59

uogbuji commented 9 months ago

I also noticed the injection risk in how we're constructing WHERE clauses here. @choccccy & I have discussed this, and he'll work on converting those to SQL query arguments. It will be a bit tricky because of the variability in arguments, so the need to manage the sequence. I'll stand by to take the task over if need be. @chimezie I'd say you can wait to review until that's done.

choccccy commented 9 months ago

ok, i think it's good to go! even got tests in for the filtered doc search options.

uogbuji commented 9 months ago

Next release is definitely feeling more & more like a 0.7.0.

uogbuji commented 9 months ago

OK @chimezie ready for review & merge to main. @choccccy I'll add you for review, too. A release can wait till Tuesday.

uogbuji commented 9 months ago

Yeah, latest change breaks the old interfaces, and is probably too much of a pain to set in a deprecation path. Someone yell at me if pulling off this bandage hurts too badly 😉

chimezie commented 9 months ago

Yeah, latest change breaks the old interfaces, and is probably too much of a pain to set in a deprecation path. Someone yell at me if pulling off this bandage hurts too badly 😉

I don't mind at all if it helps with all the things discussed and the long run

uogbuji commented 9 months ago

Just going to go ahead & merge this. We can always fix what we need in main. If needed for reversion, main tip pre merge is 2eb8a9d