Giorgi / DuckDB.NET

Bindings and ADO.NET Provider for DuckDB
https://duckdb.net
MIT License
356 stars 62 forks source link

Concurent connections problems #10

Closed travis-leith closed 2 years ago

travis-leith commented 3 years ago

I am having problems with opening multiple connections at the same time. I get the DuckDB open failed error. Works fine when I take care to open connections one by one. I think the design whereby a database object is created for the life of the process, and each connection open call takes this database object as a parameter is a simpler design to get right. It is more in line with the way DuckDb thinks under the hood, and is not contrary to anything in the ado.net spec.

Giorgi commented 3 years ago

Can you share your code that results in the error?

travis-leith commented 3 years ago

I can put together a repro in F#. Would that be OK?

Giorgi commented 3 years ago

Yes

travis-leith commented 3 years ago

You can find the repro here. I have put together 3 test cases.

If you run the project as is (setting VS to allow the exceptions) you will get a report of the test results. in test1, everything is done sequentially. If you comment out test2 and test3, it runs fine. in test2, queries are run concurrently. in test3, connections are opened concurrently, then queries are run concurrently.

The test library I am using is called Expecto. Unless you tell it not to, it runs all tests concurrently. If you leave all 3 tests, it fails. If you leave only test1, it succeeds.

travis-leith commented 3 years ago

Hmm, I just amended my test project with an implementation of the database dependency that makes sure connections are always constructed sequentially by a single thread. The problem persists if I leave it to Dapper to open the connections on separate worker threads. But if I construct AND open the connections sequentially then the problem goes away. See my branch "problem_solved".

Giorgi commented 3 years ago

@travis-leith Can you check if the ThreadSafe-Connection-Open branch solves the problem for you?

travis-leith commented 3 years ago

I am getting the error: System.AccessViolationException: 'Attempted to read or write protected memory. This is often an indication that other memory is corrupt.'

Giorgi commented 2 years ago

Fixed in #18