UpdateTool breaks Plex database

[crafty35a]

I love this tool, but for some time it has been breaking my Plex database any time I attempt to run it (all libraries show as unavailable immediately after running UpdateTool).

This could be coincidental, but the issues started immediately after I moved from Windows to UnRAID, and I did see mention in another issue of potential database corruption on UnRAID (https://github.com/mynttt/UpdateTool/issues/97#issuecomment-1310587259)

I am happy to provide any data/log info that would be helpful to getting this issue resolved, but I don't know what you need. So please let me know what info I can provide.

Thank you!

[mynttt]

Yeah, it is a very annoying issue - the problem is that the tool requires write access to the DB. Normally you can just do it programmatically from Java with the correct JDBC driver, but Plex has diverged their flavor of SQLite from the default version, so the standard JDBC implementation that exists fails here.

To circumvent it, I've tried using the SQLite binary that Plex supplies with their distribution. I might need to update that one, as an older version could cause the issues. It works on a lot of systems but on some it causes trouble which I've not been able to diagnose yet :/

• What kind of Plex container are you running, and what is the version of Plex at the moment?

• Does the corruption always happen instantaneously?

[mynttt]

So I've updated the local binary that is supplied by Plex itself to interact with their flavour of SQLite for the Docker containers now. Could you test and see if the issue persists?

And are you on amd64 or arm64 architecture?

[mynttt]

@crafty35a Another thing that you could try is setting the Docker variable USE_PLEX_SQLITE_BINARY_FOR_WRITE_ACCESS to false to check if the default JDBC driver works on your database without corruptions.

[crafty35a]

• What kind of Plex container are you running, and what is the version of Plex at the moment?

I am using lscr.io/linuxserver/plex

• Does the corruption always happen instantaneously?

I don't think it was instantaneous back when I first moved to unRAID, but it does seem to be now.

So I've updated the local binary that is supplied by Plex itself to interact with their flavour of SQLite for the Docker containers now. Could you test and see if the issue persists?

No luck - same issue after updating

And are you on amd64 or arm64 architecture?

amd64

@crafty35a Another thing that you could try is setting the Docker variable USE_PLEX_SQLITE_BINARY_FOR_WRITE_ACCESS to false to check if the default JDBC driver works on your database without corruptions.

So far, this seems to have worked! I will keep an eye on it and let you know if that changes. If I can help with anything else, let me know as well. Thanks!

[mynttt]

@crafty35a Glad it works now! I can't explain why the Plex SQLite binary usage from the docker causes issues (because it never did on my unraid) but my guess is that it has something to do with the inter-docker sharing of the database and two potential different containers. Might have to investigate that in the future..

In case it does not brick, it would be nice to get an update in a few days again whether it still works well.

[crafty35a]

In case it does not brick, it would be nice to get an update in a few days again whether it still works well.

Bad news, I woke up today to Plex not working and my Plex container is just spamming these messages non-stop:

Starting Plex Media Server. . . (you can ignore the libusb_init error) Error: Unable to set up server: sqlite3_statement_backend::loadOne: database disk image is malformed (N4soci10soci_errorE) Starting Plex Media Server. . . (you can ignore the libusb_init error) Error: Unable to set up server: sqlite3_statement_backend::loadOne: database disk image is malformed (N4soci10soci_errorE)

Looks like UpdateTool has some similar errors in it's logs from last night:

02/12/2023 10:50:54 PM [INFO ] - 2023-02-12 22:50:54 @ TaskWrapper.run: ================================================ 02/12/2023 10:50:54 PM [INFO ] - 2023-02-12 22:50:54 @ TaskWrapper.run: Starting task: imdb-docker | Execution count: 3 02/12/2023 10:50:54 PM [INFO ] - 2023-02-12 22:50:54 @ TaskWrapper.run: ================================================ 02/12/2023 10:50:54 PM [INFO ] - 2023-02-12 22:50:54 @ ImdbDockerImplementation$ImdbBatchJob.run: LIBRARIES => PRE LIBRARY FILTERING 02/12/2023 10:50:54 PM [INFO ] - 2023-02-12 22:50:54 @ ImdbDockerImplementation$ImdbBatchJob.lambda$run$0: Found library [MOVIE] UFC (ID=5) with agent: tv.plex.agents.movie and 16 item(s). 02/12/2023 10:50:54 PM [INFO ] - 2023-02-12 22:50:54 @ ImdbDockerImplementation$ImdbBatchJob.lambda$run$0: Found library [MOVIE] Movies (ID=12) with agent: tv.plex.agents.movie and 1592 item(s). 02/12/2023 10:50:54 PM [INFO ] - 2023-02-12 22:50:54 @ ImdbDockerImplementation$ImdbBatchJob.lambda$run$0: Found library [MOVIE] Movies 4k (ID=21) with agent: tv.plex.agents.movie and 32 item(s). 02/12/2023 10:50:54 PM [INFO ] - 2023-02-12 22:50:54 @ ImdbDockerImplementation$ImdbBatchJob.lambda$run$0: Found library [SERIES] TV Shows (ID=2) with agent: tv.plex.agents.series and 8779 item(s). 02/12/2023 10:50:54 PM [INFO ] - 2023-02-12 22:50:54 @ ImdbDockerImplementation$ImdbBatchJob.lambda$run$0: Found library [SERIES] TV Shows 4k (ID=22) with agent: tv.plex.agents.series and 42 item(s). 02/12/2023 10:50:54 PM [INFO ] - 2023-02-12 22:50:54 @ ImdbDockerImplementation$ImdbBatchJob.run: LIBRARIES => POST LIBRARY FILTERING 02/12/2023 10:50:54 PM [INFO ] - 2023-02-12 22:50:54 @ ImdbDockerImplementation$ImdbBatchJob.lambda$run$3: Will process library UFC (ID=5) with agent: tv.plex.agents.movie and 16 item(s). 02/12/2023 10:50:54 PM [INFO ] - 2023-02-12 22:50:54 @ ImdbDockerImplementation$ImdbBatchJob.lambda$run$3: Will process library Movies (ID=12) with agent: tv.plex.agents.movie and 1592 item(s). 02/12/2023 10:50:54 PM [INFO ] - 2023-02-12 22:50:54 @ ImdbDockerImplementation$ImdbBatchJob.lambda$run$3: Will process library Movies 4k (ID=21) with agent: tv.plex.agents.movie and 32 item(s). 02/12/2023 10:50:54 PM [INFO ] - 2023-02-12 22:50:54 @ ImdbDockerImplementation$ImdbBatchJob.lambda$run$3: Will process library TV Shows (ID=2) with agent: tv.plex.agents.series and 8779 item(s). 02/12/2023 10:50:54 PM [INFO ] - 2023-02-12 22:50:54 @ ImdbDockerImplementation$ImdbBatchJob.lambda$run$3: Will process library TV Shows 4k (ID=22) with agent: tv.plex.agents.series and 42 item(s). 02/12/2023 10:50:55 PM [ERROR] - 2023-02-12 22:50:55 @ ImdbDockerImplementation$ImdbBatchJob.run: SQLiteException exception encountered... 02/12/2023 10:50:55 PM [ERROR] - 2023-02-12 22:50:55 @ ImdbDockerImplementation$ImdbBatchJob.run: Please contact the maintainer of the application with the stacktrace below if you think this is unwanted behavior. 02/12/2023 10:50:55 PM [ERROR] - 2023-02-12 22:50:55 @ ImdbDockerImplementation$ImdbBatchJob.run: ======================================== 02/12/2023 10:50:55 PM [ERROR] - 2023-02-12 22:50:55 @ ImdbDockerImplementation$ImdbBatchJob.run: org.sqlite.SQLiteException: [SQLITE_CORRUPT] The database disk image is malformed (database disk image is malformed) 02/12/2023 10:50:55 PM at org.sqlite.core.DB.newSQLException(DB.java:1012) 02/12/2023 10:50:55 PM at org.sqlite.core.DB.newSQLException(DB.java:1024) 02/12/2023 10:50:55 PM at org.sqlite.core.DB.execute(DB.java:866) 02/12/2023 10:50:55 PM at org.sqlite.core.CoreStatement.exec(CoreStatement.java:80) 02/12/2023 10:50:55 PM at org.sqlite.jdbc3.JDBC3Statement.executeQuery(JDBC3Statement.java:68) 02/12/2023 10:50:55 PM at updatetool.common.SqliteDatabaseProvider.queryFor(SqliteDatabaseProvider.java:39) 02/12/2023 10:50:55 PM at updatetool.imdb.ImdbDatabaseSupport.updateNewAgentMetadataMapping(ImdbDatabaseSupport.java:204) 02/12/2023 10:50:55 PM at updatetool.imdb.ImdbDatabaseSupport.requestMetadata(ImdbDatabaseSupport.java:178) 02/12/2023 10:50:55 PM at updatetool.imdb.ImdbDatabaseSupport.requestEntries(ImdbDatabaseSupport.java:160) 02/12/2023 10:50:55 PM at updatetool.imdb.ImdbLibraryMetadata.fetchAll(ImdbLibraryMetadata.java:18) 02/12/2023 10:50:55 PM at updatetool.imdb.ImdbDockerImplementation$ImdbBatchJob.run(ImdbDockerImplementation.java:254) 02/12/2023 10:50:55 PM at updatetool.TaskWrapper.run(Main.java:252) 02/12/2023 10:50:55 PM at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) 02/12/2023 10:50:55 PM at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) 02/12/2023 10:50:55 PM at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) 02/12/2023 10:50:55 PM at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) 02/12/2023 10:50:55 PM at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) 02/12/2023 10:50:55 PM at java.base/java.lang.Thread.run(Unknown Source) 02/12/2023 10:50:55 PM [ERROR] - 2023-02-12 22:50:55 @ ImdbDockerImplementation$ImdbBatchJob.run: ======================================== 02/12/2023 10:50:55 PM [ERROR] - 2023-02-12 22:50:55 @ ImdbDockerImplementation$ImdbBatchJob.run: The application will terminate now.

I am going to have to restore a backup to before I started using UpdateTool again, but let me know if I can test anything for you.

[mynttt]

@crafty35a I knew it was too good to be true lol

Yeah, using JDBC to write on the database caused tons of issues after Plex changed their SQLite3 flavour from vanilla to something customized.

That is why I include their own SQLite3 binary in my container to interact with it that way for write operations in order to not corrupt the database.

What suprises me is that in some cases (like yours) this still causes issues - it is a hard to diagnose problem for sure as it does not make a lot of sense.

What I've done now is compare the SQLite3 binary used by my container with the one supplied with the linuxserver/plex version.

It is exactly the same binary even with the same md5 hash:

0ae8569d986dbddff25a246d9a220925  Plex SQLite

SQLite version 3.35.5 2021-04-19 18:32:05
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite> 

0ae8569d986dbddff25a246d9a220925  Plex SQLite

SQLite version 3.35.5 2021-04-19 18:32:05
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite> 

Sadly the API that Plex uses to update writes done by the user in the web ui is not advanced enough to write into the fields required for UpdateTool to work properly, else I would not even do writes directly to the database. So now it has shown again that the only realistic way to update anything is over their binary.

I've looked in the code again and one of the ideas I have is that the use of transactions could cause these issues.

So I would suggest the following to diagnose this, since your system already has shown that the corruption is replicable:

First I would try out leaving the transaction away and then push an update to another tag that you could try out.

If that causes the same corruption I'd say the following:

1.) I would compile a special docker container that writes the queries that are applied to the binary to a text file instead of applying them.

2.) You send me that text file over so I can analyse it.

3.) We would then try applying the queries from within the linuxserver and updatetool container to see if corruption still happens that way.

[crafty35a]

Sounds good to me, I am more than happy to help test. Let me know what tag to update to and I will report back ASAP. Should I set USE_PLEX_SQLITE_BINARY_FOR_WRITE_ACCESS back to true?

[mynttt]

@crafty35a Can you try out this container? Instead of mynttt/updatetool:latest change it to mynttt/updatetool:test

I'm curious if the issue is now fixed! And yes, for the test please set it to true - else right now it does not really matter as both options cause corruptions for you!

[mynttt]

@crafty35a And if you can recover the log for me (and remove your API keys!)

[crafty35a]

So far, nothing has broken after running with the :test tag

Here is my log (link expires in 1 hour, let me know if you don't get it in time): https://ctxt.io/2/AAAQF6PcFQ

[mynttt]

Very nice! Let's see how it goes over the next days.. Don't want to celebrate preliminary again.. I've looked at the log and it looks promising.

In case it works well on your side now I would definitly push an update so the :latest tag also profits from this.

[crafty35a]

In case it works well on your side now I would definitly push an update so the :latest tag also profits from this.

I will keep you posted!

[crafty35a]

@mynttt Bad news, looks like it died again, my Plex server is showing the same errors as yesterday.

What's odd is the UpdateTool log is showing a run one hour after the previous run yesterday and this is when it hit an error (i have it set to run every 24 hours).

02/13/2023 12:07:26 PM [INFO ] - 2023-02-13 12:07:26 @ TaskWrapper.run: ================================================ 02/13/2023 12:07:26 PM [INFO ] - 2023-02-13 12:07:26 @ TaskWrapper.run: Starting task: imdb-docker | Execution count: 1 02/13/2023 12:07:26 PM [INFO ] - 2023-02-13 12:07:26 @ TaskWrapper.run: ================================================ 02/13/2023 12:07:26 PM [ERROR] - 2023-02-13 12:07:26 @ ImdbDockerImplementation$ImdbBatchJob.run: SQLiteException exception encountered... 02/13/2023 12:07:26 PM [ERROR] - 2023-02-13 12:07:26 @ ImdbDockerImplementation$ImdbBatchJob.run: Please contact the maintainer of the application with the stacktrace below if you think this is unwanted behavior. 02/13/2023 12:07:26 PM [ERROR] - 2023-02-13 12:07:26 @ ImdbDockerImplementation$ImdbBatchJob.run: ======================================== 02/13/2023 12:07:26 PM [ERROR] - 2023-02-13 12:07:26 @ ImdbDockerImplementation$ImdbBatchJob.run: org.sqlite.SQLiteException: [SQLITE_CORRUPT] The database disk image is malformed (database disk image is malformed) 02/13/2023 12:07:26 PM at org.sqlite.core.DB.newSQLException(DB.java:1012) 02/13/2023 12:07:26 PM at org.sqlite.core.DB.newSQLException(DB.java:1024) 02/13/2023 12:07:26 PM at org.sqlite.core.DB.throwex(DB.java:989) 02/13/2023 12:07:26 PM at org.sqlite.core.NativeDB.prepare_utf8(Native Method) 02/13/2023 12:07:26 PM at org.sqlite.core.NativeDB.prepare(NativeDB.java:134) 02/13/2023 12:07:26 PM at org.sqlite.core.DB.prepare(DB.java:257) 02/13/2023 12:07:26 PM at org.sqlite.jdbc3.JDBC3Statement.executeQuery(JDBC3Statement.java:66) 02/13/2023 12:07:26 PM at updatetool.common.SqliteDatabaseProvider.queryFor(SqliteDatabaseProvider.java:39) 02/13/2023 12:07:26 PM at updatetool.common.DatabaseSupport.requestLibrary(DatabaseSupport.java:87) 02/13/2023 12:07:26 PM at updatetool.common.DatabaseSupport.requestMovieLibraries(DatabaseSupport.java:66) 02/13/2023 12:07:26 PM at updatetool.imdb.ImdbDockerImplementation$ImdbBatchJob.run(ImdbDockerImplementation.java:229) 02/13/2023 12:07:26 PM at updatetool.TaskWrapper.run(Main.java:252) 02/13/2023 12:07:26 PM at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) 02/13/2023 12:07:26 PM at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) 02/13/2023 12:07:26 PM at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) 02/13/2023 12:07:26 PM at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) 02/13/2023 12:07:26 PM at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) 02/13/2023 12:07:26 PM at java.base/java.lang.Thread.run(Unknown Source) 02/13/2023 12:07:26 PM [ERROR] - 2023-02-13 12:07:26 @ ImdbDockerImplementation$ImdbBatchJob.run: ======================================== 02/13/2023 12:07:26 PM [ERROR] - 2023-02-13 12:07:26 @ ImdbDockerImplementation$ImdbBatchJob.run: The application will terminate now. Container stopped

[mynttt]

@crafty35a Oh no.. In case you still have the corrupt DB can you send it to me?

[crafty35a]

Sure, any chance you are on Telegram? I am (deleted) - If not I will set up a container to host it real quick

[austinarchibald]

Any update here? Also getting plagued by the corrupted database problem, even with the latest 1.7.1 (have to run on docker since I have silicon mac). I've been manually refreshing metadata to update ratings for most recent movies (do nothing for TV at the moment). Thanks!

[austinarchibald]

@crafty35a You ever figure out how to run this without corrupting your database? I tried again today, set up the docker container. Corrupts it upon first run. Had to revert back. Huge bummer, which this tool worked!

[crafty35a]

@austinarchibald Unfortunately not, I was communicating with the dev for a while but there was no final resolution

@mynttt Could this issue be re-opened?

[austinarchibald]

Per the readme, I set USE_PLEX_SQLITE_BINARY_FOR_WRITE_ACCESS back to false. It did not corrupt my database on first run. A good first step, but I'll monitor and see how things go. I can see above that only worked for you temporarily. Crossing my fingers, but a little pessimistic.

Edit: Regarding my comment about episodes not being updated (but the parent show was matched and updated), it looks like most episodes were updated, but some were not. However, the show itself was, but its episodes were not updated and show no rating or badge. This seems to happen to more obscure / foreign shows upon first glance.

[mynttt]

@crafty35a @austinarchibald I have updated the Plex SQLite binary in the docker container that was kinda old. It might work now for you again with USE_PLEX_SQLITE_BINARY_FOR_WRITE_ACCESS set to true.

The corruptions don't happen on all systems and a reason for it has not to be found yet. In general using the binary has a lower risk of corruption. In case setting USE_PLEX_SQLITE_BINARY_FOR_WRITE_ACCESS to false works well then it can be used as well.

[mynttt]

@crafty35a I have tried to create a version of this tool that can be installed in an UnRaid environment without docker.

The installer script is here: https://github.com/mynttt/UpdateTool/blob/master/unraid-installer.sh - you just have to create a folder on your UnRaid system somewhere and run it there; it will then be installed in that folder.

The confg.sh acts as cfg file.

With these commands on the updatetool.sh you can start, stop, restart, update the tools version or see the current status.

Start could be added to the start up cron job so it is running automatically on startup, the files must then be stored on the boot usb drive tho as it would not work with the array. To uninstall it you can simply delete the directory you have installed it in.

Maybe this "direct" integration does not cause this issues with UnRaid as it seems to happen with Docker on some systems only.

[crafty35a]

I am going to re-enable UpdateTool now- I haven't tried it since we spoke in February. I will try USE_PLEX_SQLITE_BINARY_FOR_WRITE_ACCESS = true, then false, then the native script installer if needed. I will be sure to report back my results.

Thank you!

[austinarchibald]

@mynttt I got a notification of Plex Database being corrupted in the middle of the night. It's a different type of corruption than when I tried with USE_PLEX_SQLITE_BINARY_FOR_WRITE_ACCESS = true, which would break the server immediately and hide all affected libraries. Whereas the false corruption means you can still view/stream existing content, but you can't add any more content and if you restart the server, it won't launch again. I will rebuild the docker container with the new changes and try true again.

Update: immediately corrupts database again using true on the recent docker container. Bummer.

[mynttt]

@austinarchibald

Is there anything out of the ordinary in the logs? My first guess would be that Plex has continued from diverging from standard flavors of Sqlite which seems to cause problems when accessing the database in the way that Updatetool does.

Using their binaries was one way to circumvent this as the standard Java JDBC driver did already not work well with Plexs non-standard database anymore.

I wish there was a way through the Plex API to update the fields that are changed by Updatetool to prevent written database access at all - but to my knowledge so far their API does not support this. As those fields that need to he updated to display the proper badges can't he edited via the webinterface.

Well this is definitely a bummer. But unless the logs show any significance I'd attribute this to how Plex manages their database.

Habe you recently update a Plex version? Your system is running Unraid, right? Or has there been any change to the system as a whole?

I don't know enough about the matter but one if my guesses is that Plex might not even be at fault and something the way that Unraid manages file access between containers and storage volumes is the cause. This would be extremely hard to debug, but it could be the case as Sqlite relies heavily on the file system.

If you use Unraid you could try giving the localized installer for Unraid a try and see if running bare metal fixes this.

[mynttt]

A last resort option would be taking the image you're using for Plex now as base and creating a custom image where Updatetool is intigrated at container level in.

[austinarchibald]

@mynttt I'm not using Unraid. Standard Plex Media Center stored in ~/Library/Application Support/Plex, with the content all on an external hard drive. About 60 TB.

I don't see anything in the update tool logs, but here is the latest .log file: https://file.io/hVYnc0OzO43Q

Willing to make adjustments to how it's set up to allow for this tool (it's the only one I can find that even does it). Docker was the only method I could get to work on my silicon mac. Can't run the .jar file, but assuming it behaves the same, wouldn't it still corrupt my db? It's pretty important to get updated IMDb ratings for my library, especially TV shows!

[austinarchibald]

Quick update on my side. I disabled PMS then used this tool to automatically "repair" my database. Even though it passed all checks as OK, it still did some extraction and rebuilding. I'm not sure if it was this or something else (I've been doing quite a few things, including playing with plex-meta-manager), but when I ran UpdateTool with USE_PLEX_SQLITE_BINARY_FOR_WRITE_ACCESS = true, it did not immediately corrupt it for the first time in almost a year? I then tried running it while PMS was running, and still no corruption. I'm crossing my fingers and will keep you posted. I hope, because I really love having accurate IMDb ratings across Movies and especially Shows!

Edit: Spoke too soon. Corrupted the database a few hours later, likely once content was added. However, I was able to repair it using the same tool. Worried I might always have to do this...

[mynttt]

@austinarchibald Hmmm... That does sound really strange. I'm currently still on my master thesis and will then take some time for travel in December but I might dedicate some time to this issue at the end of January. It is really strange that these corruptions happen, but they might have changed some stuff in the database that does not immediatly cause errors in the tool itself.

[mynttt]

@austinarchibald When is it the first time that you have run into those issues? And how long have you been using this tool? I'm trying to get a better understanding of the time frame that this degraded from working to corrupting on your setup

[austinarchibald]

@mynttt Thanks. Started using it Nov 2020, upgraded to an M1 Mac in Jan 2021 (got it to work with your help), was running via macOS Terminal, then started using docker a few months later, then in Sep 2021, it began corrupting my database. It may be coincidental, but at this time time, I got a new hard drive and did a full migration of all my libraries (via Plex guide, copy all data to new hard drive, add those folders to Plex Library (so it duplicates the library, half on old drive, half on the other, then remove old folders on old hard drive, sync and empty trash). Whenever I ran the tool, it corrupted it. I would repair it constantly and eventually just stopped using it. Tried again recently (as shown in this Issues thread), but still corrupts it. There were two different kinds of corruption I noticed... sometimes it would corrupt it immediately (all libraries disappear in Plex web app), other times the server would still run (you can see libraries and stream, but when you add new content, it breaks it and all libraries hide again).

Most of the time I'd use Plex's guide for repairing a database, but more recently been using ChuckPa/PlexDBRepair. It seems to work a lot better.

[crafty35a]

Update - I have seen no corruption since I started using UpdateTool again on 11/1 (USE_PLEX_SQLITE_BINARY_FOR_WRITE_ACCESS = true)

Quick question, how does the RUN_EVERY_N_HOURS variable work? Does the tool always run on start and then again n hours later? I am just curious if that initial run could by part of the issue (e.g. on a server reboot, potentially running while the plex container is in the process of starting)

[mynttt]

@crafty35a That's good to know! I am trying to understand why it has not worked before properly and now does. Is it still working without corruption? I also expect server reboots or some concurrent write operations that are not handled gracefully due to the filesystem of the volume mount being mounted as a virtual file system on two endpoints likely have to do with the issues, but I have not yet found a way to confirm it. The thing is, on my setups, I've never encountered this issue, which means I have never been able to replicate it :D

RUN_EVERY_N_HOURS works exactly as you have imagined; the tool runs immediatly when started for the first time, and all subsequent runs are scheduled every n hour(s).

Are you still with the container or have you resorted to the installation script?

[crafty35a]

@mynttt Still no issues here, and I am running via container. I also have not rebooted my server during this time, so I can't say for sure if it could happen during plex container startup, but I can try a few reboots soon just to see if it causes any issues.

[mynttt]

@crafty35a Sounds good! Which container are you using for Plex? Might be interesting to see if the issues only arise with a specific docker image

[crafty35a]

lscr.io/linuxserver/plex and I am quite sure I have always been using this container, even when I was having issues in the past

[mynttt]

@crafty35a Is it still working without issues? I've looked into it again today and found some interesting threads regarding Unraid.

https://forums.unraid.net/bug-reports/stable-releases/sqlite-database-corruption-issues-r949/

https://forums.unraid.net/bug-reports/prereleases/sqlite-data-corruption-testing-r664/page/3/?tab=comments

I always had the guess that the writes to the database could cause this in Unraid under special scenarios. Some users have fixed it by changing the configuration of their cache disk apparently. But if it's still good for you that would be great.

@austinarchibald I've read that you're using it on a MAC with docker. I'd assume the corruptions might also have something todo with how the Docker <=> Filesystem communication is handled. Would you be willing to try it with installing Java on your MAC and running a shell script directly starting the jar? I could guide you in configuring it. How are you running Plex? Also via container or locally installed?

[austinarchibald]

@austinarchibald I've read that you're using it on a MAC with docker. I'd assume the corruptions might also have something todo with how the Docker <=> Filesystem communication is handled. Would you be willing to try it with installing Java on your MAC and running a shell script directly starting the jar? I could guide you in configuring it. How are you running Plex? Also via container or locally installed?

@mynttt Thanks! I struggled to get the jar file running in later updates, so that's why I used docker, which runs pretty seamlessly apart from the corruption. Definitely willing to try something new. Is there a specific version or method of installation for java? I typically use homebrew. And then any tips on running the jar file? Plex is run locally on the Mac Mini. Thanks!

[mynttt]

@austinarchibald The only thing of importance is that Java is >= Version 11; so whichever way you install it as long as it satisfies that requirement it should be good to go.

For running the tool I'd recommend setting it up via a script that can then be automated with a cron job.

I've released an update today that has a new ON_DEMAND capability flag which makes sense to use here to make the tool only run once and exit afterwards.

Regarding the script configuration I'll write something here tomorrow. Since you've installed Plex locally it also makes sense to use that version of Plex's sqlite binary. I'll also give you more info on that tomorrow!

[crafty35a]

@crafty35a Is it still working without issues? I've looked into it again today and found some interesting threads regarding Unraid.

Still working, but I was not running it for a while due to the recent issue where UpdateTool stopped working due to a Plex update. I did start it up again a few days ago, and have had no issues since.

Very interesting on the unRaid sqlite issues... what's weird is I have never changed the scheduler setting that they are saying is the fix for this. I wonder if I should do that?

[tsm121]

Anything new here, Is it solved? I'm running the same setup as you @crafty35a, what are your configs now?

Maybe an idea to update the README @mynttt for UnRaid if it doesn't match anymore :). E.g. I copied what was in the Docker template and I see it does miss some variables that I'm guessing has been added later (also I don't believe you can add a template like that anymore, at least not what I found. I think it needs to be added to Community Applications)

[austinarchibald]

@mynttt With your recent updates, decided to give this another try. As a reminder, I have vanilla Plex Media Server running on an M2 macOS. Been running PMS on macOS straight for around 10 years. Nothing like Unraid or anything. Then I am running UpdateTool on docker, on the same machine via Orbstack (Docker Desktop also yielded the same issues, but Orbstack is a million times better and more stable than Docker Desktop).

After first run, it doesn't seem to be damaged. I use DBRepair to check PMS integrity, repair, etc. However, it gets damaged later, either on the 2nd run 12h later, or maybe whenever a new movie/show is added (not exactly sure which, but I think the latter). Then I cannot even repair it with DBRepair and must revert to the database backup I manually make before I first ran it. Here is what DBRepair says when trying to repair it:

Runtime error near line 3177486: UNIQUE constraint failed: activities.id (19) / this error repeated dozens of times for many lines
Error 1 from Plex SQLite while importing from './dbtmp/library.plexapp.sql-2024-08-04_07.38.48'
Cannot continue.

Would love your help to get this resolved. Love your tool, and love how it updates IMDB ratings (especially for recent content which can change so drastically after its added and especially for TV shows/episodes). I've also, in the past, tried running via a custom script you wrote for me, which runs it in a terminal window. I'd prefer docker though. Can share any log files or whatever for more info.

Thanks!