[Bug]: Audiobookshelf keeps blowing out all user edited metadata and covers

[Nitrousoxide]

Describe the issue

Every few days I come back to my audiobookshelf docker install to find that it has reverted the entire audiobook database to an unedited state after doing a scan. I believe the "watcher" scan is reverting all book states to whatever metadata can be gleaned from the directory path and any covers that are in the audiobook directory rather than the metadata fields in the database. I have reverted back to a previous database state and turned off the watcher so I guess we'll see if it still repeats.

I'm not sure what the best way to handle this is. I could obviously disable the watcher, and that's probably fine. I'm not trying to initialize a new audiobook instance anymore from existing directory. But if the intention of the watcher is to overwrite existing metadata with file paths info and the content of the actual file directory you probably want flag that toggle with a warning saying it can be destructive to your audiobook library's metadata.

For me this was pretty destructive since it blew out 300+ audiobooks' metadata. Luckily I keep good backups via dupliciti so I could restore a previous day's state before hours of my work managing the metadata were blown out.

Before the metadata was nuked

Here's a snippet of the log as it destroyed it:

"timestamp":"2023-09-28T00:01:45.237Z","message":"setChapters: Using embedded chapters in first audio file /audiobooks/Mike Duncan/The Storm Before the Storm/The Storm Before the Stormê    The Beginning of the End of the Roman Republic [B0747WH1RK].m4b","levelName":"DEBUG","level":1}
{"timestamp":"2023-09-28T00:01:45.272Z","message":"Success saving abmetadata to \"/metadata/items/be7e76fd-2554-4005-95a9-4971ed2f181a/metadata.abs\"","levelName":"DEBUG","level":1}
{"timestamp":"2023-09-28T00:01:45.272Z","message":"Library item \"Brandon Sanderson/Yumi and the Nightmare Painter\" key \"ino\" changed from \"15689227\" to \"18315371\"","levelName":"DEBUG","level":1}
{"timestamp":"2023-09-28T00:01:45.273Z","message":"Library item \"Brandon Sanderson/Yumi and the Nightmare Painter\" changed: [ino,size,lastScan]","levelName":"DEBUG","level":1}
{"timestamp":"2023-09-28T00:01:45.507Z","message":"setChapters: Using embedded chapters in first audio file /audiobooks/Brandon Sanderson/Yumi and the Nightmare Painter/Yumi_and_the_Nightmare_Painter_by_Brandon_Sanderson.m4b","levelName":"DEBUG","level":1}
{"timestamp":"2023-09-28T00:01:45.528Z","message":"Success saving abmetadata to \"/metadata/items/ce123821-1593-45c5-8263-4ddd51d8cb18/metadata.abs\"","levelName":"DEBUG","level":1}
{"timestamp":"2023-09-28T00:01:45.528Z","message":"Library item \"Pirateabe/The Wandering Inn/The Wind Runner\" key \"ino\" changed from \"15690702\" to \"18313943\"","levelName":"DEBUG","level":1}
{"timestamp":"2023-09-28T00:01:45.528Z","message":"Library item \"Pirateabe/The Wandering Inn/The Wind Runner\" changed: [ino,size,lastScan]","levelName":"DEBUG","level":1}
{"timestamp":"2023-09-28T00:01:45.886Z","message":"setChapters: Using embedded chapters in first audio file /audiobooks/Pirateabe/The Wandering Inn/The Wind Runner/The Wind Runner꞉ The Wandering Inn, Book 10 [B0C82TS539].m4b","levelName":"DEBUG","level":1}
{"timestamp":"2023-09-28T00:01:45.886Z","message":"Updating book description \"The floodwaters have receded, and the war is over. A ruler has fallen. The Wind Runner lives, missing yet not forgotten. The world is changing, growing larger. After the horrors of war and the loss of spring comes the renewal of summer, a chance for a new beginning. Ryoka Griffin has reappeared in Izril's north. The Quarass is dead, yet the King of Destruction's war continues. In the jungles of Baleros, Geneva and her friends fall under the shadow of the Titan of Baleros. Amidst it all, Erin Solstice grieves for the dead and pulls her family close. Yet there is always a door leading her onwards to a magical future...\" => \"The world is changing, growing larger. After the horrors of war and the loss of spring comes the renewal of summer, a chance for a new beginning. Ryoka Griffin has reappeared in Izril's north. The Quarass is dead, yet the King of Destruction's war continues. In the jungles of Baleros, Geneva and her friends fall under the shadow of the Titan of Baleros. Amidst it all, Erin Solstice grieves for the dead and pulls her family close. Yet there is always a door leading her onwards to a magical future...\" for book \"The Wind Runner\"","levelName":"DEBUG","level":1}
{"timestamp":"2023-09-28T00:01:45.886Z","message":"Updating book genres \"Science Fiction & Fantasy\" => \"Action & Adventure, Epic, Paranormal\" for book \"The Wind Runner\"","levelName":"DEBUG","level":1}
{"timestamp":"2023-09-28T00:01:45.908Z","message":"Updating book \"The Wind Runner\" added author \"Pirateabe\"","levelName":"DEBUG","level":1}
{"timestamp":"2023-09-28T00:01:45.925Z","message":"Updating book \"The Wind Runner\" removed author \"pirateaba\"","levelName":"DEBUG","level":1}
{"timestamp":"2023-09-28T00:01:45.955Z","message":"Success saving abmetadata to \"/metadata/items/1378482e-0542-452d-af65-b51d53a815f3/metadata.abs\"","levelName":"DEBUG","level":1}
{"timestamp":"2023-09-28T00:01:45.972Z","message":"Removed 1 authors","levelName":"INFO","level":2}

After everything was nuked

The server settings

Steps to reproduce the issue

1. Have a set of audiobooks that you've added to the server
2. Edit some metadata, such as adding covers or changing names
3. Enable the "watcher" scanner
4. Wait for the scanner to run on its normal schedule
5. Your edited metadata will be overwritten

Audiobookshelf version

2.4.3

How are you running audiobookshelf?

Docker

[advplyr]

Metadata should not be getting overwritten (with your server settings) unless you have a metadata.json or metadata.abs file inside the same folder of your books. The watcher doesn't run on a schedule, it will only get triggered when it detects a file has been added or removed from a library folder. Then it will only run the scan on the existing book or new book.

There is an issue relating to covers getting re-extracted from audio files on scans which will explain why covers would be reverted https://github.com/advplyr/audiobookshelf/issues/2110. I've already fixed that for the next release. I'm not aware of any other details getting overwritten

[advplyr]

I just looked through your logs and it appears you are having an issue with your files changing their inode values. Can you share what OS and file system you are using for your books?

Related issue on inodes https://github.com/advplyr/audiobookshelf/issues/1447 but not the same since those are short inodes having collisions

[Nitrousoxide]

The container itself is using docker, so whatever base OS that uses.

I have a remote smb/cifs share mounted to the container

volumes:
  - /srv/remotemount/Media/Media/Audiobooks:/audiobooks

which points to a Synology network share (and thus is on a BTRFS file system)

[advplyr]

Do you have scheduled library scans enabled? When pressing edit on a library and going to schedule

[Nitrousoxide]

Do you have scheduled library scans enabled? When pressing edit on a library and going to schedule

Yep

[Nitrousoxide]

I just changed my scan schedule to run now as opposed to midnight and it did nuke all my metadata again. So I think it's that not the watcher. I exported my log out too if you want to see it. 2023-09-28_6cb590d6-567f-4f4d-8f88-9f772f630783.txt

[advplyr]

I found and fixed the issue here. This is specific to not storing the metadata file with the item and the reason you were running up against it with all of your media is because your file system is changing the inode value. I'll release the patch for the this over the weekend.

This will resolve the issue of metadata being overwritten but the scanner will still report that all of your media got updated because of that inode change. I don't know much about your specific setup but maybe you can configure some setting so that the inode values don't keep changing?

As a side note, the watcher probably won't work for you because of the network file system. That is why scheduled library scans were added so it's normal, just giving an fyi

[Nitrousoxide]

Is there a way for the user to switch to storing the metadata with the item? I did try flipping that, but it didn't seem to do anything (at least immediately) so I assume it only does it for new items going forward? If it writes the metadata to the /audiobook file from /metadata at a scheduled time a note in the mouseover might be a good addition. If not bringing up a popup to start the migration might be a good addition.

In my case the faster storage is on the local drive rather than the network share, so I'll probably keep the metadata local to the server for better performance.

I'll turn off scans for the library for now until this update is pushed, and I update for now as a workaround.

[advplyr]

The metadata file is written to the new location the next time the details are updated for that book. There is a feature request open I believe for automatically moving everything over. With the upcoming patch it won't matter whether you store it with the items or not so probably best to keep it as you had.

[advplyr]

Fixed in v2.4.4