[Bug]: "Clean-up database" seems to be broken

[ummon29]

What happened?

Database entry's that should have been purged from the database with the 'Clean-up database' reappear when the files are scanned again in a different location of the music folder.

Steps to reproduce

1. Delete files from your music folder.
2. scan media folders
3. Watch database entry's disappear as expected.
4. Clean-up database
5. Add files back to a different sub-folder of your music folder
6. scan media folders again
7. expected behavior would be that the files in the new location would appear in the database. In reality the new working database entry's appear as expected but all previous entry's for the files reappear too. Pointing to nonexistent file locations.

This was working in previous versions. I'm not entirely sure when it broke. I'm sure it was working in 11.0.2. I noticed it was broken using 11.1.3 stable (when reorganizing my folder structure).

Version

11.1.4-SNAPSHOT (Edge)

Version Detail

11.1.4-SNAPSHOT.20240628143437

Operating System

Linux (RaspberryPi OS)

Java Version

OpenJDK 64-Bit Server VM (build 17.0.11+9-Debian-1deb12u1, mixed mode, sharing)

Database

Other

DB Detail

Build in HSQL database

Configuration paramter

EncryptionKeyPassword=xxx EncryptionKeySalt=xxx SettingsChanged=1717107256878 JWTKey=xxx IndexCreationInterval=7 IndexCreationHour=4 FastCacheEnabled=true OrganizeByFolderStructure=true UploadsFolder=%{['USER_MUSIC_FOLDERS'][0]}/Incoming ExcludePattern= IgnoreSymLinks=false IndexString=A B C D E F G H I J K L M N O P Q R S T U V W X-Z(XYZ) IgnoredArticles=The El La Los Las Le Les Der Die Das GenreSeparators=; Shortcuts=New Incoming Podcast PlaylistFolder=/var/playlists MusicFileTypes=mp3 ogg oga aac m4a m4b flac wav wma aif aiff ape mpc shn mka opus alm 669 mdl far xm mod fnk imf it liq wow mtm ptm rtm stm s3m ult dmf dbm med okt emod sfx m15 mtn amf gdm stx gmc psm j2b umx amd rad hsc flx gtk mgt mtp wv VideoFileTypes=flv avi mpg mpeg mp4 m4v mkv mov wmv ogv divx m2ts webm CoverArtFileTypes2=cover.jpg cover.png cover.gif folder.jpg jpg jpeg gif png CoverArtSource=TAGFILE CoverArtConcurrency=4 CoverArtQuality=90 SortAlbumsByYear=true GettingStartedEnabled=false WelcomeTitle=Welcome to Airsonic! WelcomeSubtitle= WelcomeMessage2= LoginMessage= Theme=default LocaleLanguage=en LocaleCountry= LocaleVariant= spring.liquibase.parameters.mysqlVarcharLimit=384 DbBackupUpdateInterval=168 DbBackupRetentionCount=2 server.servlet.session.timeout=2h HideIndexedFiles=true EnableCueIndexing=true FullScan=false ClearFullScanSettingAfterScan=false

Proxy Server

No response

client detail

For example Firefox and Audinaut

language

English

Relevant log output

2024-06-30 23:54:33.440 INFO --- o.a.p.c.MusicFolderSettingsController : Cleaning database...
2024-06-30 23:54:33.440 INFO --- o.a.p.c.MusicFolderSettingsController : Deleting non-present cover art...
2024-06-30 23:54:42.538 INFO --- o.a.p.c.MusicFolderSettingsController : Deleting non-present artists...
2024-06-30 23:54:42.544 INFO --- o.a.p.c.MusicFolderSettingsController : Deleting non-present albums...
2024-06-30 23:54:42.603 INFO --- o.a.p.c.MusicFolderSettingsController : Deleting non-present media files...
2024-06-30 23:54:44.632 INFO --- o.a.p.c.MusicFolderSettingsController : Deleting non-present media folders...
2024-06-30 23:54:44.634 INFO --- o.a.p.c.MusicFolderSettingsController : Refreshing playlist stats...
2024-06-30 23:54:45.017 INFO --- o.a.p.c.MusicFolderSettingsController : Database cleanup complete.

[kagemomiji]

@ummon29 I have created fixed image. Could you try ghcr.io/kagemomiji/airsonic-advanced:pr-538

[ummon29]

Is there a simple way to get the war file?. I'm not using a docker setup here.

[kagemomiji]

@ummon29 It is available from here. war file in zip

[ummon29]

Had time to try. Sadly the behavior is still exactly the same as before.

[kagemomiji]

@ummon29

I have not succeeded to reproduce this issue. I would like to know more details.

Delete files from your music folder.

Where are the files located? Are they directly under the Music Folder, or are they organized in a directory structure like MusicFolder/Artist/Album/files?

In reality the new working database entry's appear as expected but all previous entry's for the files reappear too. Pointing to nonexistent file locations.

How does this issue appear on the web browser GUI? Should I understand that music files placed in a different folder can still be accessed through the previous folder structure? Could you share me some screenshots?

[yuygfgg]

I have a similar problem. In my case, clean-database simply does nothing. Even I remove all files in folder, no entries are deleted in db.

[ummon29]

The file structure is something like: Music/Artist/s/Sepultura/1993 Chaos A.D./files I changed this to: Music/Artist/s/Sepultura/1993 Chaos A.D. (Reissue 1996)/files

Access through the file structure works:

Search shows the old path: clicking on the result leads to the new directory and everything works as shown above

Access through an App (in this case Audinaut) shows duplicates and only one version plays. The other one tries without success to access toe old location.

I tested different Apps without any different results. Substreamer for example.

Changing the path again would lead to tree entries for the Apps etc.

[kagemomiji]

@ummon29 Thank you!

Airsonic Advanced has exhibited the following behavior since the original repository:

The album name and artist name displayed in the music file list are obtained from the ID3 tags. They are not derived from the folder structure. Similarly, the album names displayed in the search results use the ID3 tag album names of the files within the folder, not the folder names. Several Android client apps utilize an API that performs searches based on ID3 tags to retrieve information. Therefore, even if the folder structure is modified, the paths will not be updated unless the ID3 tags are corrected.

To modify the displayed album name, you need to either click the Edit icon or use a music management application to update the ID3 tags. Therefore, please try editing the ID3 tags.

[ummon29]

I'm relatively sure it's not the ID3 Tags. The files are not shown when I remove them from the file system I experimented with not only putting them to a different directory. I also edited the ID3 Tags and changed the Album name. It didn't work and only resulted in more duplicates in Audinaut and other Apps. I removed the whole file system from the file systems to be scanned. Did a Full scan. Did a cleanup afterwards. The Web-UI and all Apps showed no files afterward. A soon as I did put the file system back in the configuration and did a new scan everything including the duplicates was back. The only way I found to fix it was to completely start over and delete the whole database. After that all duplicates where gone. But it's a lot of work to create the users again etc. Everything else I tried didn't work. It also wasn't a defective database. After starting with a newly setup Airsonic-Advanced i could reproduce the same behavior again. The only thing I couldn't verify was if it is maybe a defective file that breaks the cleanup of the database. Sadly I didn't find anything in the logs.

I checked the Tags. and everything looked fine as far as I can tell. I'm using kid3 to edit the tags. I normally use only ID3v2.3.0 tags with an entry for Albuminterpret and Interpret since I found it the best way to keep Albums with different Interprets as one pice by setting an common Albuminterpret like 'various artists'.

[kagemomiji]

@ummon29 I have created fixed war file here Could you try again?

[ummon29]

Yay. Looks like it's fixed now. :-) Thank you.