mixxxdj / mixxx

Mixxx is Free DJ software that gives you everything you need to perform live mixes.
http://mixxx.org
Other
4.43k stars 1.27k forks source link

Incomplete anlysis, waveform overview and display, of a song file, after conducting library "Analyze" #11280

Open nPrevail opened 1 year ago

nPrevail commented 1 year ago

Screenshot from 2023-02-15 23-56-44

I analyzed my library of files, and in some instances, some of my files would be analyzed but incompletely scanned. This image attached to this message is what I see.

When playing the incomplete song file, the audio cuts off before the song reaches the end and the deck stops playing.

I have tried "hiding" the song file, deleting the song file data from the library, and I have tried reanalyzing the song file, yet it rescans and still shows as incomplete at the same time stamp.

There is nothing wrong with the song file itself; it plays fine in a music player (ie. VLC)

I happen to have backups of my music files; back ups of the same songs in other hard drives. The only solution I found was to use the back up of the song file from the directory folder, and overwrite the original song from my Mixxx music folder.

I have tried duplicating the song file in Mixxx library/folder (copy and pasting the song file), and it does not fix the issue. But grabbing the song file, which had a different "created by" date, and a slightly different file size (unedited metadata?) seemed to fix the issue.

Perhaps this is a metadata song file issue?

uklotzde commented 1 year ago

Decoding stops at the first unrecoverable error. Please check the logs for more information.

nPrevail commented 1 year ago

Please check the logs for more information.

What's the fastest way to check the logs? I'm look through the .mixxx folder, and I have about 10 log files to look at.

EDIT: How do I check logs? Would it be something like this from the manual? https://manual.mixxx.org/2.3/en/chapters/appendix/commandline_dev_tools.html

uklotzde commented 1 year ago

Start Mixxx, reset the waveform data of the affected song, then load it on a deck. The log of the current session is in mixxx.log.

uklotzde commented 1 year ago

Starting mixxx from a terminal and simply watching the outputs is another option.

Please also send me one of the affected files. It would be interesting to investigate what's wrong and which decoder (libmad or ffmpeg) fails to decode it.

ronso0 commented 1 year ago

@nPrevail Yes, could also run Mixxx from the command line with --developer argument. The only benefit is that you'd see the debug output in real time, not just when it's flushed to disk

uklotzde commented 1 year ago

The --developer argument is only needed if Mixxx crashes. Otherwise it just causes unnecessary I/O.

nPrevail commented 1 year ago

Start Mixxx, reset the waveform data of the affected song, then load it on a deck.

I have tried resetting the waveform. I actually reset all the parameters, and had loaded onto the deck, and the issue would still persist.

It's unfortunate that none of these logs are timestamped, but I'll paste all the relevant log lines I have.

uklotzde commented 1 year ago

Resetting the waveform enforces the analyzer to re-decode the whole file for getting the logs. The error will not be fixed magically.

uklotzde commented 1 year ago

Bugs that fix themselves under unknown circumstances would be the worst case. Reproducible bugs are good bugs ;)

nPrevail commented 1 year ago

Logs

Debug [AnalyzerThread 0 #1]: AnalyzerThread - Analyzing QFileInfo(/home/nPrev/Music/DJ (LL)/Disco/disco edits and covers/Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 02 Mercy.flac) Debug [AnalyzerThread 0 #1]: AnalysisDAO fetched 2 analyses, 5252991 bytes for track 3147 in 121 ms Debug [AnalyzerThread 0 #1]: Reading waveform from byte array: allSignalSize 610918 visualSampleRate 441 audioVisualRatio 100 Debug [AnalyzerThread 0 #1]: Reading waveform from byte array: allSignalSize 3842 visualSampleRate 2.77197 audioVisualRatio 15909.3 Debug [AnalyzerThread 0 #1]: AnalyzerWaveform - loadStored - Stored waveform loaded Debug [AnalyzerThread 0 #1]: AnalyzerBeats preference settings: Plugin: "qm-tempotracker:0" Fixed tempo assumption: true Re-analyze when settings change: true Re-analyze imported from other software: true Fast analysis: false Debug [AnalyzerThread 0 #1]: AnalyzerKey preference settings: Plugin: "keyfinder:2" Re-analyze when settings change: true Fast analysis: false Debug [AnalyzerThread 0 #1]: Keys version/sub-version unchanged since previous analysis. Not analyzing. Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Debug [Main]: GLWaveformRendererSignalShader::loadShaders Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Warning [AnalyzerThread 0 #1]: SoundSourceFLAC - FLAC decoding error "STREAM_DECODER_ERROR_STATUS_FRAME_CRC_MISMATCH" in file "/home/nPrev/Music/DJ (LL)/Disco/disco edits and covers/Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 02 Mercy.flac" Warning [AnalyzerThread 0 #1]: SoundSourceFLAC - FLAC decoding error "STREAM_DECODER_ERROR_STATUS_LOST_SYNC" in file "/home/nPrev/Music/DJ (LL)/Disco/disco edits and covers/Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 02 Mercy.flac" Warning [AnalyzerThread 0 #1]: SoundSourceFLAC - Unexpected frame index 20172800 > 20168704 while decoding FLAC file "/home/nPrev/Music/DJ (LL)/Disco/disco edits and covers/Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 02 Mercy.flac" Warning [AnalyzerThread 0 #1]: AudioSource - Failed to read sample frames: expected = [20168704 -> 20172800) , actual = [20168704 -> 20168704) Info [AnalyzerThread 0 #1]: AudioSource - Shrinking readable frame index range: before = [0 -> 30545828) , after = [0 -> 20168704) Debug [AnalyzerThread 0 #1]: ReplayGain 2.0 (libebur128) result is -9.37333 dB for QFileInfo(/home/nPrev/Music/DJ (LL)/Disco/disco edits and covers/Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 02 Mercy.flac) Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Resuming analysis Debug [Main]: TrackAnalysisScheduler - Resuming Info [Main]: AnalysisFeature - Resuming analysis Debug [Main]: TrackAnalysisScheduler - Resuming Debug [Main]: Successfully deserialized BeatGrid Debug [Main]: Successfully deserialized KeyMap Debug [Main]: BaseTrackCache(0x562e329e55a0) updateIndexWithQuery took 0 ms Debug [Main]: BaseTrackPlayerImpl::slotLoadTrack "[Channel1]" Track(0x562e32fd33d0) Debug [Main]: TrackAnalysisScheduler - Resuming Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Debug [Main]: TrackCollectionManager - Saving track "/home/nPrev/Music/DJ (LL)/Disco/disco edits and covers/Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 02 Mercy.flac" in internal collection Debug [Main]: AnalyzerThread - Enqueueing next track 3146 Debug [AnalyzerThread 0 #1]: AnalyzerThread - Dequeued next track 3146 Debug [AnalyzerThread 0 #1]: AnalyzerThread - Analyzing QFileInfo(/home/nPrev/Music/DJ (LL)/Disco/disco edits and covers/Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 01 Set It Out.flac) Debug [AnalyzerThread 0 #1]: AnalysisDAO fetched 2 analyses, 3889488 bytes for track 3146 in 32 ms Debug [AnalyzerThread 0 #1]: Reading waveform from byte array: allSignalSize 453256 visualSampleRate 441 audioVisualRatio 100 Debug [AnalyzerThread 0 #1]: Reading waveform from byte array: allSignalSize 3842 visualSampleRate 3.73618 audioVisualRatio 11803.5 Debug [AnalyzerThread 0 #1]: AnalyzerWaveform - loadStored - Stored waveform loaded Debug [AnalyzerThread 0 #1]: AnalyzerBeats preference settings: Plugin: "qm-tempotracker:0" Fixed tempo assumption: true Re-analyze when settings change: true Re-analyze imported from other software: true Fast analysis: false Debug [AnalyzerThread 0 #1]: AnalyzerKey preference settings: Plugin: "keyfinder:2" Re-analyze when settings change: true Fast analysis: false Debug [AnalyzerThread 0 #1]: Keys version/sub-version unchanged since previous analysis. Not analyzing. Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Debug [AnalyzerThread 0 #1]: ReplayGain 2.0 (libebur128) result is -6.6405 dB for QFileInfo(/home/nPrev/Music/DJ (LL)/Disco/disco edits and covers/Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 01 Set It Out.flac) Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Resuming analysis Debug [Main]: TrackAnalysisScheduler - Resuming Info [Main]: AnalysisFeature - Resuming analysis Debug [Main]: TrackAnalysisScheduler - Resuming Debug [Main]: Successfully deserialized BeatGrid Debug [Main]: Successfully deserialized KeyMap Debug [Main]: BaseTrackCache(0x562e329e55a0) updateIndexWithQuery took 0 ms Debug [Main]: BaseTrackPlayerImpl::slotLoadTrack "[Channel1]" Track(0x562e48334340) Debug [Main]: TrackAnalysisScheduler - Resuming Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Debug [Main]: TrackCollectionManager - Saving track "/home/nPrev/Music/DJ (LL)/Disco/disco edits and covers/Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 01 Set It Out.flac" in internal collection Debug [Main]: AnalyzerThread - Enqueueing next track 3147 Debug [AnalyzerThread 0 #1]: AnalyzerThread - Dequeued next track 3147 Debug [AnalyzerThread 0 #1]: AnalyzerThread - Analyzing QFileInfo(/home/nPrev/Music/DJ (LL)/Disco/disco edits and covers/Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 02 Mercy.flac) Debug [AnalyzerThread 0 #1]: AnalysisDAO fetched 2 analyses, 5252991 bytes for track 3147 in 28 ms Debug [AnalyzerThread 0 #1]: Reading waveform from byte array: allSignalSize 610918 visualSampleRate 441 audioVisualRatio 100 Debug [AnalyzerThread 0 #1]: Reading waveform from byte array: allSignalSize 3842 visualSampleRate 2.77197 audioVisualRatio 15909.3 Debug [AnalyzerThread 0 #1]: AnalyzerWaveform - loadStored - Stored waveform loaded Debug [AnalyzerThread 0 #1]: AnalyzerBeats preference settings: Plugin: "qm-tempotracker:0" Fixed tempo assumption: true Re-analyze when settings change: true Re-analyze imported from other software: true Fast analysis: false Debug [AnalyzerThread 0 #1]: AnalyzerKey preference settings: Plugin: "keyfinder:2" Re-analyze when settings change: true Fast analysis: false Debug [AnalyzerThread 0 #1]: Keys version/sub-version unchanged since previous analysis. Not analyzing. Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Warning [AnalyzerThread 0 #1]: SoundSourceFLAC - FLAC decoding error "STREAM_DECODER_ERROR_STATUS_FRAME_CRC_MISMATCH" in file "/home/nPrev/Music/DJ (LL)/Disco/disco edits and covers/Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 02 Mercy.flac" Warning [AnalyzerThread 0 #1]: SoundSourceFLAC - FLAC decoding error "STREAM_DECODER_ERROR_STATUS_LOST_SYNC" in file "/home/nPrev/Music/DJ (LL)/Disco/disco edits and covers/Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 02 Mercy.flac" Warning [AnalyzerThread 0 #1]: SoundSourceFLAC - Unexpected frame index 20172800 > 20168704 while decoding FLAC file "/home/nPrev/Music/DJ (LL)/Disco/disco edits and covers/Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 02 Mercy.flac" Warning [AnalyzerThread 0 #1]: AudioSource - Failed to read sample frames: expected = [20168704 -> 20172800) , actual = [20168704 -> 20168704) Info [AnalyzerThread 0 #1]: AudioSource - Shrinking readable frame index range: before = [0 -> 30545828) , after = [0 -> 20168704) Debug [AnalyzerThread 0 #1]: ReplayGain 2.0 (libebur128) result is -9.37333 dB for QFileInfo(/home/nPrev/Music/DJ (LL)/Disco/disco edits and covers/Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 02 Mercy.flac) Info [Main]: AnalysisFeature - Suspending analysis Debug [Main]: TrackAnalysisScheduler - Suspending Info [Main]: AnalysisFeature - Resuming analysis Debug [Main]: TrackAnalysisScheduler - Resuming Info [Main]: AnalysisFeature - Resuming analysis Debug [Main]: TrackAnalysisScheduler - Resuming Debug [AnalyzerThread 5 #10]: Writing waveform from byte array: dataSize 358134 allSignalSize 358134 visualSampleRate 441 audioVisualRatio 100 Debug [AnalyzerThread 3 #8]: Writing waveform from byte array: dataSize 238712 allSignalSize 238712 visualSampleRate 441 audioVisualRatio 108.844 Debug [AnalyzerThread 6 #11]: Writing waveform from byte array: dataSize 416126 allSignalSize 416126 visualSampleRate 441 audioVisualRatio 100 Debug [AnalyzerThread 5 #10]: AnalysisDAO saved analysis 20762 "2958701 (1380325 compressed)" bytes for track 6874 in 425 ms Debug [AnalyzerThread 5 #10]: Saved waveform analysis for trackId 6874 analysisId 20762 Debug [AnalyzerThread 5 #10]: Writing waveform from byte array: dataSize 3842 allSignalSize 3842 visualSampleRate 4.72851 audioVisualRatio 9326.4 Debug [AnalyzerThread 5 #10]: AnalysisDAO saved analysis 20763 "31420 (15337 compressed)" bytes for track 6874 in 13 ms Debug [AnalyzerThread 5 #10]: Saved waveform summary analysis for trackId 6874 analysisId 20763 Debug [AnalyzerThread 5 #10]: AnalyzerWaveform - Waveform generation for track 6874 done 13 s

uklotzde commented 1 year ago

The FLAC decoder is trustworthy, the last bug has been fixed long ago. Unfortunately, some of your files seem to be corrupt.

If you start vlc from the command line you might see warning logs. Maybe they ignore the error and continue decoding, accepting a short discontinuity in the decoded audio that might be unnoticeable. Mixxx is more rigorous here.

nPrevail commented 1 year ago

The main culprit file is: /home/nPrev/Music/DJ (LL)/Disco/disco edits and covers/Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 02 Mercy.flac"

nPrevail commented 1 year ago

Unfortunately, some of your files seem to be corrupt.

Ah I see... I wonder how that happened...

uklotzde commented 1 year ago

Use the official FLAC tools to analyze the file contents. Should be possible somehow. If they report that the file is ok then we need to check our FLAC decoder.

uklotzde commented 1 year ago

And don't delete or modify the file! Keep it until the case could be closed.

nPrevail commented 1 year ago

I have the "tainted" file on another drive. I can probably tinker with FLAC tools to analyze the contents for corruption.

nPrevail commented 1 year ago

Can anyone point out which FLAC tools I can use?

I've installed mingw64-flac-tools and mingw64-flac, but I don't know how to use it...

Screenshot from 2023-02-16 14-01-22

@uklotzde

nPrevail commented 1 year ago

So, I ran flac --analyze on the song file, and got this:

[nPrev@fedora disco edits and covers]$ 

flac 'Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 02 Mercy.flac' --analyze

flac 1.3.4
Copyright (C) 2000-2009  Josh Coalson, 2011-2016  Xiph.Org Foundation
flac comes with ABSOLUTELY NO WARRANTY.  This is free software, and you are
welcome to redistribute it under certain conditions.  Type `flac' for details.

Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 02 Mercy.flac: *** Got error code 2:FLAC__STREAM_DECODER_ERROR_STATUS_FRAME_CRC_MISMATCH

Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 02 Mercy.flac: ERROR while decoding data
                                                                      state = FLAC__STREAM_DECODER_ABORTED

I guess we can confirm that I just happen to have a corrupted file? I have no idea how it happened though...

uklotzde commented 1 year ago

Now you need to find out what caused this corruption. Mixxx could only be involved if TagLib corrupts FLAC files when writing metadata.

nPrevail commented 1 year ago

Now you need to find out what caused this corruption.

Got it.

Well, some good news (for now). I ran this bash script I found on github, found all the corrupted tracks, fixed all the corrupted tracks (by running the script that pretty much self-reencoded flac to flac), and it seemed to fix all the corruption!

I used this script: https://github.com/cgomesu/bash-flac-diag

Worked perfectly! I'm glad to know it can be resolved in case it happens again.

So this time around, a clean slate will help me figure out what's causing the issue.

daschuer commented 1 year ago

Great news. Which type of issue did the script report?

The script uses flac -st in the first place: https://github.com/cgomesu/bash-flac-diag/blob/b5a06fa03367e30c9bea929c569912693709d1e3/flac_diag.sh#L161

What would be that appropriate behavior of Mixxx that we can avoid future bug reports like this?

daschuer commented 1 year ago

When may investigate which tests the FLAC tool does and compare it with the Mixxx implemtation.

nPrevail commented 1 year ago

Which type of issue did the script report?

I'll talk about what I did and the behavior I saw from the script.

The script came with four separate bash scripts: flac diagnosis for good or bad songs, flac fix, flac remove, and install (installs all related tools necessary for FLAC). I used everything except "install.sh" and "bad_flac_remover.sh"

The "flac_diag.sh" script indicated if the song file was bad or good. Within the script's directory, the script creates a folder called "log," and then the script creates two logs: one for bad, the other for good. The script also creates a directory called "errors,"and that directory created individual error logs for each bad song file.

I then ran "bad_flac_fixer.sh" to fix all the corrupted tracks mentioned from the log directory. It pretty much "self re-encoded the corrupt files as if they were new again (encoding from FLAC to FLAC, and same file source). The best part is that I didn't need to locate any directories as the script found and logged the directories of the corrupt files.

Overall, for scanning and diagnosing 4,000+ tracks, the process took about 60 minutes to analyze. I had about 70 bad FLAC files. Fixing the flac files probably took about 10 minutes. Obviously this will vary based on computer specs.

More details on how and what the script does can be found in the readme, and it's pretty straight forward.

Here are some screen shots: Screenshot from 2023-02-17 17-14-28 Screenshot from 2023-02-17 17-14-19 Screenshot from 2023-02-17 17-11-33 Screenshot from 2023-02-17 17-11-25 Screenshot from 2023-02-17 17-22-57

What would be that appropriate behavior of Mixxx that we can avoid future bug reports like this?

It'd be neat if we could implement this type of script into Mixxx. Like a "Diagnose" and "Fix" button from the "Analyze" section in Mixxx.

Unfortunately, it seems to only currently work with FLAC files, and depending on a person's filesystem and configuration, there may be some prerequisites needed to ensure the FLAC tools work (however, the script's readme states that most modern distros should already have these FLAC tools installed by default).

nPrevail commented 1 year ago

Okay, I thought this issue was over, but I discovered a few more things that also conflict in Mixxx: ID3v2 tags

This error causes the same error I mentioned: incomplete waveform analysis for songs with this type of error.

As I remove the ID3v2 tags from these songs, what type of tag is recommended to be used in Mixxx?

From the same FLAC fixing script, I received the following outputs. I bolded the errors that are causing issues, and I have to fix these issues in some other method because this script can't remove ID3v2 tags:

.............. .............. Starting fixer for the bad flac files... .............. Re-encoding file: /home/nPrev/Music/DJ (LL)/Disco/in the zone/09 Always There.flac

flac 1.3.4 Copyright (C) 2000-2009 Josh Coalson, 2011-2016 Xiph.Org Foundation flac comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it under certain conditions. Type `flac' for details.

ERROR: input file /home/nPrev/Music/DJ (LL)/Disco/in the zone/09 Always There.flac has an ID3v2 tag Done. Now try playing the file again.

.............. Re-encoding file: /home/nPrev/Music/DJ (LL)/Electronic dance music/Deep house/out there/03 Glow & Glare.flac

flac 1.3.4 Copyright (C) 2000-2009 Josh Coalson, 2011-2016 Xiph.Org Foundation flac comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it under certain conditions. Type `flac' for details.

ERROR: input file /home/nPrev/Music/DJ (LL)/Electronic dance music/Deep house/out there/03 Glow & Glare.flac has an ID3v2 tag Done. Now try playing the file again. .............. .............. Re-encoding file: /home/nPrev/Music/DJ (LL)/Electronic dance music/Techno/getting started/04 The Sugarcubes - Motorcrash [Justin Robertson Mix].flac

flac 1.3.4 Copyright (C) 2000-2009 Josh Coalson, 2011-2016 Xiph.Org Foundation flac comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it under certain conditions. Type `flac' for details.

ERROR: input file /home/nPrev/Music/DJ (LL)/Electronic dance music/Techno/getting started/04 The Sugarcubes - Motorcrash [Justin Robertson Mix].flac has an ID3v2 tag Done. Now try playing the file again. .............. .............. Re-encoding file: /home/nPrev/Music/DJ (LL)/Parties/Ballroom/Disco Swing/01. Unknown - Title 1.flac

flac 1.3.4 Copyright (C) 2000-2009 Josh Coalson, 2011-2016 Xiph.Org Foundation flac comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it under certain conditions. Type `flac' for details.

ERROR: input file /home/nPrev/Music/DJ (LL)/Parties/Ballroom/Disco Swing/01. Unknown - Title 1.flac has an ID3v2 tag Done. Now try playing the file again.

uklotzde commented 1 year ago

This should not be a Mixxx issue. ID3v2 tags in FLAC files are supported as a fallback by most software but not recommended.

https://xiph.org/flac/faq.html#general__tagging

Please consult the official docs when in doubt.

nPrevail commented 1 year ago

I ran Mixxx via terminal so I could see a live log. I reset the Mixxx file data (BPM, waveform, key, and etc.) with the ID3v2 issue, and I received this output:

`Warning [AnalyzerThread 1 #2]: SoundSourceFLAC - FLAC decoding error "STREAM_DECODER_ERROR_STATUS_LOST_SYNC" in file "/home/nPrev/Music/DJ (LL)/Disco/in the zone/09 Always There.flac"
Warning [AnalyzerThread 1 #2]: SoundSourceFLAC - FLAC decoding error "STREAM_DECODER_ERROR_STATUS_LOST_SYNC" in file "/home/nPrev/Music/DJ (LL)/Disco/in the zone/09 Always There.flac"
Warning [AnalyzerThread 1 #2]: SoundSourceFLAC - Unexpected frame index 7442432 > 7434240 while decoding FLAC file "/home/nPrev/Music/DJ (LL)/Disco/in the zone/09 Always There.flac"
Warning [AnalyzerThread 1 #2]: AudioSource - Failed to read sample frames: expected = [7434240 -> 7438336) , actual = [7434240 -> 7434240)

I know we've seen this error before, but I guess this script is unable to fix/re-encode the FLAC file because of the ID3v2 tag: ERROR: input file /home/nPrev/Music/DJ (LL)/Disco/in the zone/09 Always There.flac **has an ID3v2 tag**

I'll read the general tagging link you sent.

nPrevail commented 1 year ago

The other strange thing too is that this song file 09 Always There.flac came from a compilation of other songs downloaded from Bandcamp, yet this song seems to be the only one affected in the bunch...

This song file plays fine in VLC, but it prevents Mixxx from fully scanning the waveform.

uklotzde commented 1 year ago

If the FLAC decoder fails to parse the byte stream from your files with non-standard tags this is still not a Mixxx issue. We can't do anything about it.

nPrevail commented 1 year ago

From what I learned, it's difficult to remove ID3v2 tags, and I did find a script to fix MP3 files (and maybe FLAC), but fortunately, I have backup copies of these songs, and these backups seem to work fine. Just wanted to share the possible solution in case folks accidentally have ID3v2 data in their FLAC files.

script: https://github.com/tillt/CleanID3

robbert-vdh commented 1 year ago

Is the waveform truncated or does it not line up with the actual track (so, it ends up being stretched). I've been seeing a similar issue on the main branch where track waveforms are sometimes compressed like this: (this is deck A doubled to deck B):

image

Usually when it happens when loading a new file, it only happens the first time the file is loading. Reloading it in the same deck afterwards fixes the waveform. There's nothing in the log at the standard verbosity level, so I'll need to check if anything useful shows up in a trace log.

nPrevail commented 1 year ago

It's usually truncated, and also the track stops playing when it reaches the abrupt end (not playing the complete track from beginning to end, but stopping at the gap, like your Deck B's waveform).

After fixing it, the waveform still shows up truncated, but like you said, once I reset and re-analyze the song file, the waveform is analyzed correctly and is no longer truncated. HOWEVER, that only happens after I fix my track corruption, which I had a handful of corrupted FLAC files.

uklotzde commented 1 year ago

It's usually truncated, and also the track stops playing when it reaches the abrupt end (not playing the complete track from beginning to end, but stopping at the gap, like your Deck B's waveform).

This is not the case here when you look at the pictures. The waveform is just wrongly scaled, but otherwise complete on both decks. It's the same track, too.

nPrevail commented 1 year ago

This is not the case here when you look at the pictures.

Ah you're right! My apologies for not seeing that the first time.

From my original picture of my post, the song and waveform would instantly cut off and would stop playing. After fixing the corrupt track, it would play. After resetting and analyze the waveform, the waveform would analyze fine.