unable to load or analyze track with strange characters in filename

[mixxxbot]

Reported by: Be-ing Date: 2018-04-05T05:23:43Z Status: Won't Fix Importance: Medium Launchpad Issue: lp1761376

---

I tested starting Mixxx with a fresh configuration directory on my old laptop. When I went to batch analyze my library, no tracks would analyze at all. I noticed some tracks with no metadata detected by Mixxx at the top of the track table. They are an album of an artist with non-ASCII characters in their name (Søren Nordström). Other tracks analyzed fine, but if any of these tracks were selected for batch analysis, none of them would analyze. ls showed the file names as

'01 - S+'$'\251''ren Nordstr+'$'\302''m - Qualia.flac'
'02 - S+'$'\251''ren Nordstr+'$'\302''m - Deep State.flac'
'03 - S+'$'\251''ren Nordstr+'$'\302''m - Teleological Attractor.flac'
'04 - S+'$'\251''ren Nordstr+'$'\302''m - Trinity.flac'
'05 - S+'$'\251''ren Nordstr+'$'\302''m - Novelty.flac'
'06 - S+'$'\251''ren Nordstr+'$'\302''m - Behold A Pale Horse.flac'
'07 - S+'$'\251''ren Nordstr+'$'\302''m - Ghost-Dubbing.flac'

'08 - S+'$'\251''ren Nordstr+'$'\302''m - Monad.flac'
The mplayer and flac command line programs worked with these files fine, so their data is not corrupted -- just the filenames.

I'm not sure where those weird characters came from. The files are on an ext4 filesystem, but I think they may have been copied to a FAT filesystem on an external drive then back to ext4 at some point. I redownloaded the original files from Ektoplasm and this is what ls shows:

'01 - Søren Nordström - Qualia.flac' '02 - Søren Nordström - Deep State.flac' '03 - Søren Nordström - Teleological Attractor.flac' '04 - Søren Nordström - Trinity.flac' '05 - Søren Nordström - Novelty.flac' '06 - Søren Nordström - Behold A Pale Horse.flac' '07 - Søren Nordström - Ghost-Dubbing.flac' '08 - Søren Nordström - Monad.flac'

Mixxx loads and analyzes those files fine. I tried copying them to a FAT formatted flash drive and back to my ext4 hard drive and the filenames remained okay and Mixxx could load them.

Here is what the log shows when I try to analyze one of the files with a weird filename:

Debug [Main]: DlgAnalysis(0x13938cb0, name = "DlgAnalysis") analysisActive true 
Debug [AnalyzerQueue 6]: AnalyzerQueue - Entering thread 
Debug [AnalyzerQueue 6]: DbConnection - Opening database connection "MIXXX-8" QSqlDatabase(driver=""QSQLITE"", database=""/home/be/mixxx-config-test/mixxxdb.sqlite"", host=""localhost"", port=-1, user=""mixxx"", open=false) 
Debug [AnalyzerQueue 6]: DbConnectionPool - Cloned thread-local database connection "MIXXX-8" QSqlDatabase(driver=""QSQLITE"", database=""/home/be/mixxx-config-test/mixxxdb.sqlite"", host=""localhost"", port=-1, user=""mixxx"", open=true) 
Debug [AnalyzerQueue 6]: DbConnectionPooled - Found thread-local database connection "MIXXX-8" QSqlDatabase(driver=""QSQLITE"", database=""/home/be/mixxx-config-test/mixxxdb.sqlite"", host=""localhost"", port=-1, user=""mixxx"", open=true) 
Debug [AnalyzerQueue 6]: AnalyzerQueue - Analyzing "" "/home/be/music/yes/Soren Nordstrom - Deep State - 2014 - FLAC/01 - S+�ren Nordstr+�m - Qualia.flac" 
Warning [AnalyzerQueue 6]: SoundSourceProxy - Unable to decode file "" 
Warning [AnalyzerQueue 6]: AnalyzerQueue - Failed to open file for analyzing: "/home/be/music/yes/Soren Nordstrom - Deep State - 2014 - FLAC/01 - S+�ren Nordstr+�m - Qualia.flac" 
Debug [Main]: DlgAnalysis(0x13938cb0, name = "DlgAnalysis") analysisActive false 
Debug [AnalyzerQueue 6]: DbConnection - Closing database connection: "MIXXX-8" QSqlDatabase(driver=""QSQLITE"", database=""/home/be/mixxx-config-test/mixxxdb.sqlite"", host=""localhost"", port=-1, user=""mixxx"", open=true) 
Debug [AnalyzerQueue 6]: AnalyzerQueue - Exiting thread 
Debug [Main]: AnalyzerWaveform - ~AnalyzerWaveform(): 
Debug [Main]: DlgAnalysis(0x13938cb0, name = "DlgAnalysis") analysisActive false

If I try to load them to a deck, I get an error dialog that says:

The file '/home/be/music/yes/Soren Nordstrom - Deep State - 2014 - FLAC/06 - S+�ren Nordstr+�m - Behold A Pale Horse.flac' could not be found.

In that case the log shows:
Debug [Main]: GlobalTrackCache - Resolving track by id 1145 
Debug [Main]: GlobalTrackCache - Cache miss - allocating track ["/home/be/music/yes/Soren Nordstrom - Deep State - 2014 - FLAC/06 - S+�ren Nordstr+�m - Behold A Pale Horse.flac" | "" | 1145] 
Debug [Main]: GlobalTrackCache - Cache miss - inserting new track into cache ["/home/be/music/yes/Soren Nordstrom - Deep State - 2014 - FLAC/06 - S+�ren Nordstr+�m - Behold A Pale Horse.flac" | "" | 1145] QObject(0x0)  
Debug [Main]: BeatFactory::loadBeatsFromByteArray could not parse serialized beats. 
Debug [Main]: BaseTrackCache(0x2eea900) updateIndexWithQuery took 0 ms 
Debug [Main]: BaseTrackPlayerImpl::slotLoadTrack "[Channel1]" 
Debug [CachingReaderWorker 1]: CachingReaderWorker - "[Channel1]" loadTrack() load failed for" "/home/be/music/yes/Soren Nordstrom - Deep State - 2014 - FLAC/06 - S+�ren Nordstr+�m - Behold A Pale Horse.flac" ", unlocked reader lock 
Debug [Main]: WCoverArt::slotCoverFound WCoverArt(0x22dc0e20, name = "LibraryCoverArt")  "CoverInfoRelative(NONE,UNKNOWN,,0x0)" QSize(0, 0) 
Debug [Main]: BaseTrackPlayerImpl::slotTrackLoaded 
Debug [Main]: stray BaseTrackPlayerImpl::slotTrackLoaded() 
Debug [Main]: Failed to load track "/home/be/music/yes/Soren Nordstrom - Deep State - 2014 - FLAC/06 - S+�ren Nordstr+�m - Behold A Pale Horse.flac" "The file '/home/be/music/yes/Soren Nordstrom - Deep State - 2014 - FLAC/06 - S+�ren Nordstr+�m - Behold A Pale Horse.flac' could not be found." 
Debug [Main]: BaseTrackPlayerImpl::slotTrackLoaded

[mixxxbot]

Commented by: uklotzde Date: 2018-04-05T09:00:03Z

---

I don't think that this is a bug in Mixxx. If file names contain invalid multi-byte character sequences that Qt is not able to parse correctly and falls back to inserting placeholders for unrecognized glyphs we can't do anything about it.

[mixxxbot]

Commented by: daschuer Date: 2018-04-05T09:53:32Z

---

Some times ago I have did a detailed analysis of a similar bug. Unfortunately I cannot find it right now. Result was exactly as like Uwe assumes.

A workaround would be to handle all paths in the native encoding. But that is not supported by QT, so there is no chance to fix this.

[mixxxbot]

Commented by: Be-ing Date: 2018-04-05T11:10:16Z

---

In that case, I agree it's okay if Mixxx can't load it. But the batch analyzer should not fail to analyze okay tracks if one of these tracks with a corrupted filename is included in the analysis.

[mixxxbot]

Issue closed with status Won't Fix.