asg017 / sqlite-vec

A vector search SQLite extension that runs anywhere!
Apache License 2.0
4.3k stars 136 forks source link

Database is locked error when inserting or after creating virtual table #40

Closed hkfi closed 4 months ago

hkfi commented 4 months ago

There seems to be an issue with sqlite-vec, where running an insert to the vector virtual table seems to cause it to get stuck.

This occurs when:

I have a reproducible example in this repo.

dleviminzi commented 4 months ago

Can you isolate the example from the rest of your code base and post it here?

asg017 commented 4 months ago

@hkfi can you upgrade to 0.0.1-alpha.37 and try again?

I think this had to do with #19 , where vec0 tables would open statements and not finalize them across transactions, which would prevent vacuums and introduce problems when using multiple connections.

I think that's what was happening here - sqlx sometimes uses multiple connections at once, so if 1 connection had a vec0 table that wrote results and didn't clear up open statements, then another connection would fail waiting for it.

Please upgrade and let me know if it works. Alternatively, make sure you run INSERTs in transactions (ex BEGIN then COMMIT at the end), or tell sqlx to only use a single connection at a time. That's not ideal, so I hope upgrading to 0.0.1-alpha.37 works by itself.

hkfi commented 4 months ago

@asg017 Yes, that sounds right. 0.0.1-alpha.37 seems to have fixed the issue. Thanks!