mixxxbot
commented
2 years ago Commented by: daschuer Date: 2012-09-15T19:44:45Z
See: http://www.sqlite.org/faq.html#q6
This is an other argument for Bug #6384, a dedicated library thread.
Open mixxxbot opened 2 years ago
mixxxbot
commented
2 years ago Commented by: daschuer Date: 2012-09-15T19:44:45Z
See: http://www.sqlite.org/faq.html#q6
This is an other argument for Bug #6384, a dedicated library thread.
mixxxbot
commented
2 years ago Commented by: kain88-de Date: 2012-09-15T20:12:11Z
To solve the our library backends (DAOs) should be able to talk to the LibraryScanner(LS). I imagine this to look something like this.
DAO -> request LS to pause if running LS -> pauses any running scan and signal DAO that the scanner has paused and closed all open transactions DAO -> receives ok signal from LS and then commits transaction DAO -> signals LS that it's finished it's transactions and tells it to unpause LS -> reassumes scan if paused before
This should be necessary even if the LS and the library run in the same thread. The implementation of this scheme should be relatively straight forward but I won't have time to do this in the next couple weeks
mixxxbot
commented
2 years ago mixxxbot
commented
2 years ago Commented by: rryan Date: 2012-11-21T01:35:53Z
This is much reduced in severity now that the locking issues caused by the setlog feature are fixed. A better fix should come in a future release.
mixxxbot
commented
2 years ago mixxxbot
commented
2 years ago Commented by: rryan Date: 2012-11-21T16:15:26Z
Many people (generally developers) were experiencing database-is-locked error messages and crashes because of the way we were listing Playlists on the sidebar. We were using a QSqlTableModel which if there are more rows than it fetches by default in the result set then it keeps a lock on the database. Since we were never fetching more results from it the lock was always on once the playlist table reached a certain size. The setlog feature triggered this because it created a new playlist every time you start Mixxx. Fixing this issue got rid of the database-is-locked issues that were plaguing us over the summer.
On Wed, Nov 21, 2012 at 4:19 AM, Max Linke
Which locks are you referring to?
-- You received this bug notification because you are a member of Mixxx Development Team, which is subscribed to Mixxx. https://bugs.launchpad.net/bugs/1051106
Title: libraryscanner blocks the library for other transactions
To manage notifications about this bug go to: https://bugs.launchpad.net/mixxx/+bug/1051106/+subscriptions
mixxxbot
commented
2 years ago Commented by: kain88-de Date: 2012-11-21T16:47:12Z
Ah now I remember but they don't affect this bug at all. It is still not possible to change playlists/crates while a library scan is running
mixxxbot
commented
2 years ago Commented by: ulatekh Date: 2013-04-12T15:59:02Z
Yes, it IS possible to interact with the Mixxx window while a library scan is running. The problem is that the library-scan dialog is not modal. A call to "setWindowModality (Qt::WindowModal)" in the LibraryScannerDlg constructor is part of the solution. Making the dialog visible immediately, instead of after 2 seconds (done in LibraryScannerDlg::slotUpdate()) is the other part of the solution.
mixxxbot
commented
2 years ago Commented by: rryan Date: 2013-04-12T16:31:03Z
Hm, some people have gigantic libraries and making mixxx unusable during the scan via modal dialog would be bad. I like the idea of periodic commits of the scanner if only so that people who have large libraries can start to use the library before the scan is finished. I would prefer we fix locking issues by switching all library access to a single thread.
On Fri, Apr 12, 2013 at 11:59 AM, Steven Boswell
Yes, it IS possible to interact with the Mixxx window while a library scan is running. The problem is that the library-scan dialog is not modal. A call to "setWindowModality (Qt::WindowModal)" in the LibraryScannerDlg constructor is part of the solution. Making the dialog visible immediately, instead of after 2 seconds (done in LibraryScannerDlg::slotUpdate()) is the other part of the solution.
-- You received this bug notification because you are a member of Mixxx Development Team, which is subscribed to Mixxx. https://bugs.launchpad.net/bugs/1051106
Title: libraryscanner blocks the library for other transactions
To manage notifications about this bug go to: https://bugs.launchpad.net/mixxx/+bug/1051106/+subscriptions
mixxxbot
commented
2 years ago Commented by: ulatekh Date: 2013-04-22T15:54:09Z Attachments: mixxx-01-incrementalrescan.tar.bz2
The enclosed patch implements incremental/pausable rescan. Unpack this to your mixxx checkout directory (i.e. the one with .bzr, .bzrignore, and mixxx in it). It'll unpack a patch file at the top level, and add mixxx/src/dlglibraryscanner.ui . Let me know what you think!
mixxxbot
commented
2 years ago Commented by: ulatekh Date: 2013-05-02T01:45:16Z
As far as I know, my patch doesn't change the multithreading behavior of library-rescan -- it remains as before, just with the possibility of incrementally committing changes. No?
mixxxbot
commented
2 years ago Commented by: daschuer Date: 2013-05-12T22:46:04Z
My comment #11 was only a related link to this bug. No issue with patch #10. Sorry for the confusion.
mixxxbot
commented
2 years ago Commented by: quequotion Date: 2013-06-28T07:46:48Z
I have this problem compounded by another issue: mixxx apparently never saves the database or it's configuration, and so rescans the library and asks me to configure outputs on every startup.
The result is often that I will get but 870128 and mixxx will crash on open or that it will not crash but never open....
mixxxbot
commented
2 years ago Commented by: quequotion Date: 2013-06-28T07:49:54Z
*bug 870128
Also, by "not crash but never open" i mean the process is running, no errors come up, nothing output to terminal either and it uses 96% cpu (about 1 hyperthread of 8 or 1/2 of a core actually). Although my desktop doesn't slow down at all and everything looks fine, mixxx seems to be working very hard on something it never finishes and then no interface ever appears on the screen.
This happens inconsistently. Sometimes it opens with no problem (but still rescans the library and requires configuration).
mixxxbot
commented
2 years ago Commented by: daschuer Date: 2013-06-28T08:32:05Z
Hi quequotion
Thank you for reporting tis bug. It sound that it is a different issue with not saving the DB. There might be the same root cause, but as long it is not verified, I would prefer to track your issue separate.
So can you please file a new bug and put all infos in it that might be helpful, including the Mixxx version, your OS + Version and your mixxx.log file from faulty starts.
Thank you.
mixxxbot
commented
2 years ago Commented by: quequotion Date: 2013-12-13T15:21:10Z
Daniel, by "compounded by another issue" I mean to say that I do in fact have /this/ problem, as well as an additional problem which makes this one worse.
The other, separate issue, is that Mixxx usually needs to be completely reconfigured at each run. I haven't reported a bug for this because I am uncertain how to debug the issue. The secondary issue is inconsistent, with no particular relationship to any set of circumstances as far as I can tell.
Because of the secondary problem, wherein the library has to be completely rescanned at almost every run, I experience /this/ problem at almost every run (database locked).
Actually, I haven't looked into this for a while and I am not certain of the current status of either issue.
mixxxbot
commented
2 years ago Commented by: rryan Date: 2014-01-19T15:32:53Z
Bumping out of 1.12.0 -- a more sound fix is on the way for 1.13 (e.g. Nazar's library thread).
mixxxbot
commented
2 years ago Commented by: kentarveolsen Date: 2014-07-18T09:34:15Z
Wishlist: ...
Configurable DB Backend, so there is the possibility of Team sharing the data with the option of DropBox type of file sharing and so on..
mixxxbot
commented
2 years ago mixxxbot
commented
2 years ago Commented by: rryan Date: 2014-11-13T21:24:35Z
The scanner has 3 phases:
1) Opening: legacy library import, mark all tracks/dirs as needs_verification
2) recursive parallel scan -- updates LibraryHases and inserts new tracks into library table
3) Final cleanup -- verify tracks outside library directories, mark non-verified tracks/dirs as deleted, detect moved files
Now that we have switched to not rolling back new track insertions upon cancel I think it makes sense to not put phase 2 in a transaction.
The scanner is the only DB connection that touches the library hashes and needs_verification bits so there is no race condition caused by writing/reading those outside of a transaction since there is only one scanner.
This will allow other parts of Mixxx to work for most of the time the scanner is running. Thoughts?
mixxxbot
commented
2 years ago Commented by: kain88-de Date: 2014-12-01T18:02:41Z
But keeping step 2 as one transaction is the reason why the library scan is so fast now. If we use serveral transaction then the libraryscanner will be bound by the time it takes sqlite to complete a single transaction, which is really slow due to a lot of error/consistency checking of sqlite.
I'm also not sure about the race-conditions. AFAIK SQLite supports only one transaction at a time and will fail others during an active transaction. So what happens if we start a transaction in the UI thread the same time as a transaction happens in the libraryscanner thread. One of them will fail. It will be possible to recover from that and to retry the transaction in the scanner but there is no straigtforward way to do this in the UI.
I'm still in favor of keeping Phase 2 in a transaction.
mixxxbot
commented
2 years ago Commented by: uklotzde Date: 2022-01-01T09:11:38Z
At least rescanning the library on startup should be disabled:
https://mixxx.discourse.group/t/auto-dj-randomly-stopping/23919/9
Reported by: kain88-de Date: 2012-09-14T23:06:25Z Status: Confirmed Importance: Critical Launchpad Issue: lp1051106 Tags: library, scanner Attachments: mixxx-01-incrementalrescan.tar.bz2
The libraryScanner runs in a separated thread with an open transaction which blocks the sqlite database. So if you've added a large chunk of new music and scan it during that time it is not possible to create/change/delete playlists/crates/etc.
I'm not sure of the right solution for this. We could write some threadsafe transaction class or move the library and scanner into a thread that is seperated from the GUI and save a queue of transactions in that thread.