search

m-barthelemy / vapor-queues-fluent-driver

A Fluent implementation for https://github.com/vapor/queues (Vapor4)
MIT License
32 stars 16 forks source link

Comment out transaction commit in SQLITE pop #10

Closed thuotdwz closed 4 years ago

thuotdwz commented 4 years ago

This will allow SQLITE for use in unit testing Fixes #9

m-barthelemy commented 4 years ago

I think the COMMIT here shouldn't be uncommented, since there's no transaction involved.

Instead, the Sqlite code should probably be changed and wrapped into a transaction like what is done in MySQLPopQuery.swift. If you have the time and willingness to try updating the PR and test it, I'll happily approve and merge the change!

thuotdwz commented 4 years ago

Unfortunately if done in transaction app will hang when using in process worker, I am not sure why this happens :(

m-barthelemy commented 4 years ago

That's exactly why SQLite is not supported. To work properly the SQL statements must be wrapped in a transaction, otherwise multiple Queues worker would pickup the same job.

The problem is, when running it in process or a separate worker, you get 2xCpuCores Queues workers, all trying to poll SQLitefor new jobs to run at the same time, which causes a deadlock. SQLite has very limited support for concurrent requests.

The only way SQLite could possibly work is if you only have 1 single Queue worker.

thuotdwz commented 4 years ago

actually I tried with wal mode (PRAGMA journal_mode=WAL;) and single in process worker and seems it is working now with transaction!

so I copied over transaction from mysql as you suggested before and updated the patch