mackron / dr_libs

Audio decoding libraries for C/C++, each in a single source file.
Other
1.27k stars 206 forks source link

dr_flac: `runtime error: left shift of negative value` errors with clang ubsan #128

Closed u3shit closed 4 years ago

u3shit commented 4 years ago

Download the flac file from here: https://opengameart.org/content/a-journey-awaits I'm using the simple_playback example from miniaudio to try this, but I could also reproduce it in my other project that uses dr_flac directly. I've updated dr_flac to 0.12.9 on miniaudio master then compiled the simple_playback demo:

$ clang-10 -fsanitize=undefined ../simple_playback.c -o simple_playback -ldl -lm -lpthread
$ ./simple_playback /mnt/ram/A\ Journey\ Awaits.flac 
Press Enter to quit...../../extras/dr_flac.h:8828:48: runtime error: left shift of negative value -104
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8828:48 in 
../../extras/dr_flac.h:8829:48: runtime error: left shift of negative value -4428
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8829:48 in 
../../extras/dr_flac.h:9089:51: runtime error: left shift of negative value -406
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9089:51 in 
../../extras/dr_flac.h:9090:51: runtime error: left shift of negative value -22
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9090:51 in 
../../extras/dr_flac.h:9094:51: runtime error: left shift of negative value -834
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9094:51 in 
../../extras/dr_flac.h:9095:51: runtime error: left shift of negative value -790
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9095:51 in 
../../extras/dr_flac.h:9268:52: runtime error: left shift of negative value -22
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9268:52 in 
../../extras/dr_flac.h:9269:52: runtime error: left shift of negative value -58
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9269:52 in 
../../extras/dr_flac.h:8670:48: runtime error: left shift of negative value -2861
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8670:48 in 
../../extras/dr_flac.h:8671:48: runtime error: left shift of negative value -1280
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8671:48 in 

OS: gentoo linux, clang 10.0.0. I get the same errors with gcc 9.3.0's ubsan too. Since these errors mostly came from #if defined(DRFLAC_SUPPORT_SSE2), I tried to disable simd, I got even more errors:

$ clang-10 -fsanitize=undefined ../simple_playback.c -o simple_playback -ldl -lm -lpthread -DDR_FLAC_NO_SIMD
$ ./simple_playback /mnt/ram/A\ Journey\ Awaits.flac
Press Enter to quit...../../extras/dr_flac.h:8775:53: runtime error: left shift of negative value -2
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8775:53 in
../../extras/dr_flac.h:8776:53: runtime error: left shift of negative value -2
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8776:53 in
../../extras/dr_flac.h:8773:53: runtime error: left shift of negative value -5
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8773:53 in
../../extras/dr_flac.h:8774:53: runtime error: left shift of negative value -3
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8774:53 in
../../extras/dr_flac.h:8770:53: runtime error: left shift of negative value -1
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8770:53 in
../../extras/dr_flac.h:8771:53: runtime error: left shift of negative value -3
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8771:53 in
../../extras/dr_flac.h:8768:53: runtime error: left shift of negative value -4
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8768:53 in
../../extras/dr_flac.h:8769:53: runtime error: left shift of negative value -2
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8769:53 in
../../extras/dr_flac.h:8794:48: runtime error: left shift of negative value -104
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8794:48 in
../../extras/dr_flac.h:8795:48: runtime error: left shift of negative value -4428
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8795:48 in
../../extras/dr_flac.h:8937:56: runtime error: left shift of negative value -1
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8937:56 in
../../extras/dr_flac.h:8938:56: runtime error: left shift of negative value -54
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8938:56 in
../../extras/dr_flac.h:8939:56: runtime error: left shift of negative value -83
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8939:56 in
../../extras/dr_flac.h:8953:38: runtime error: left shift of negative value -98
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8953:38 in
../../extras/dr_flac.h:8954:38: runtime error: left shift of negative value -154
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8954:38 in
../../extras/dr_flac.h:8957:38: runtime error: left shift of negative value -14
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8957:38 in
../../extras/dr_flac.h:8958:38: runtime error: left shift of negative value -118
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8958:38 in
../../extras/dr_flac.h:8959:38: runtime error: left shift of negative value -176
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8959:38 in
../../extras/dr_flac.h:8936:56: runtime error: left shift of negative value -217
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8936:56 in
../../extras/dr_flac.h:8951:38: runtime error: left shift of negative value -422
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8951:38 in
../../extras/dr_flac.h:8952:38: runtime error: left shift of negative value -688
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8952:38 in
../../extras/dr_flac.h:8956:38: runtime error: left shift of negative value -444
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8956:38 in
../../extras/dr_flac.h:8944:56: runtime error: left shift of negative value -2
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8944:56 in
../../extras/dr_flac.h:8942:56: runtime error: left shift of negative value -2
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8942:56 in
../../extras/dr_flac.h:8943:56: runtime error: left shift of negative value -2
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8943:56 in
../../extras/dr_flac.h:8941:56: runtime error: left shift of negative value -6
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8941:56 in
../../extras/dr_flac.h:9018:47: runtime error: left shift of negative value -406
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9018:47 in
../../extras/dr_flac.h:9019:47: runtime error: left shift of negative value -22
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9019:47 in
../../extras/dr_flac.h:9023:53: runtime error: left shift of negative value -417
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9023:53 in
../../extras/dr_flac.h:9024:53: runtime error: left shift of negative value -395
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9024:53 in
../../extras/dr_flac.h:9232:53: runtime error: left shift of negative value -2
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9232:53 in
../../extras/dr_flac.h:9231:53: runtime error: left shift of negative value -3
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9231:53 in
../../extras/dr_flac.h:9229:53: runtime error: left shift of negative value -8
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9229:53 in
../../extras/dr_flac.h:9230:53: runtime error: left shift of negative value -7
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9230:53 in
../../extras/dr_flac.h:9225:53: runtime error: left shift of negative value -2
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9225:53 in
../../extras/dr_flac.h:9226:53: runtime error: left shift of negative value -4
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9226:53 in
../../extras/dr_flac.h:9227:53: runtime error: left shift of negative value -4
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9227:53 in
../../extras/dr_flac.h:9224:53: runtime error: left shift of negative value -6
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9224:53 in
../../extras/dr_flac.h:9245:52: runtime error: left shift of negative value -22
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9245:52 in
../../extras/dr_flac.h:9246:52: runtime error: left shift of negative value -58
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9246:52 in
../../extras/dr_flac.h:8612:52: runtime error: left shift of negative value -393
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8612:52 in
../../extras/dr_flac.h:8613:52: runtime error: left shift of negative value -248
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8613:52 in
../../extras/dr_flac.h:8610:52: runtime error: left shift of negative value -387
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8610:52 in
../../extras/dr_flac.h:8611:52: runtime error: left shift of negative value -287
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8611:52 in
../../extras/dr_flac.h:8616:52: runtime error: left shift of negative value -139
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8616:52 in
../../extras/dr_flac.h:8617:52: runtime error: left shift of negative value -518
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8617:52 in
../../extras/dr_flac.h:8618:52: runtime error: left shift of negative value -380
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8618:52 in
../../extras/dr_flac.h:8615:52: runtime error: left shift of negative value -442
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8615:52 in
../../extras/dr_flac.h:8636:39: runtime error: left shift of negative value -2861
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8636:39 in
../../extras/dr_flac.h:8637:39: runtime error: left shift of negative value -1280
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8637:39 in

Strictly speaking, shifting a negative number is implementation defined (https://wiki.sei.cmu.edu/confluence/display/c/INT13-C.+Use+bitwise+operators+only+on+unsigned+operands), but it should work correctly on msvc and gcc according to the above link. To get rid of this warning, we should either have to cast those integers to unsigned, or use __attribute__((no_sanitize("shift-base"))) annotation on the functions that do bit shifting.

mackron commented 4 years ago

Does it actually sound right when running through simple_playback? I ran it through my automated test program which uses libFLAC as a benchmark and it's passing everything. That is, it's output is identical to libFLAC. Some of those numbers (-442, -2861, etc.) would result in blatantly wrong output so I'm not sure about this.

I'm not convinced yet that this is a bug in dr_flac. It looks like those shifts are from lines like this?

unusedBitsPerSample + pFlac->currentFLACFrame.subframes[0].wastedBitsPerSample;

Here, wastedBitsPerSample cannot be negative since it's an unsigned uint8, so it's not that causing it. That leaves unusedBitsPerSample as the culprit, and that is calculated as 32 - pFlac->bitsPerSample. That means pFlac->bitsPerSample would need to be greater than 32 for it to become negative. However, this should never happen because pFlac->bitsPerSample is set from a lookup table with this code:

const drflac_uint8 bitsPerSampleTable[8] = {0, 8, 12, (drflac_uint8)-1, 16, 20, 24, (drflac_uint8)-1};   /* -1 = reserved. */
...
header->bitsPerSample = bitsPerSampleTable[bitsPerSample];

And the -1 cases are guarded with this:

if (bitsPerSample == 3 || bitsPerSample == 7) {
    continue;
}

So considering dr_flac is producing identical output to libFLAC and I'm not seeing any obvious logic errors, I'm not convinced (yet) that this is a dr_flac bug. I also tried explicitly checking for the case where pFlac->bitsPerSample is > 32 and nowhere did it get triggered. This is running on MSVC and GCC on Windows by the way (haven't been able to run it with -fsanitize=undefined).

Is it somehow possible to break on one of those errors and post a stack trace (I've never used the ubsan stuff so I'm not sure if this is possible or not)? That may be handy to see exactly which variables are out of place.

Is it happening with just that one file by the way?

kcgen commented 4 years ago

Reproduced in dosbox-staging compiled using gcc 9.2.1 with -fsanitize=address,undefined -fsanitize-recover=signed-integer-overflow, running OpenCublicPlayer acting as a CD-DA player, with dosbox simulating an audio CDROM composed of underly FLAC files.

2020-04-09_21-27

Is it happening with just that one file by the way?

Tried a handful of FLACs (some I've locally encoded using FLAC 1.3.3 w/ 1-second seek-points), and all trigger this.

mackron commented 4 years ago

I need to see a stack trace and the content of the drflac object at the time of these errors before I'll even look at it because I don't know what to look for. Does it actually sound correct - no glitches or anything? Maybe some memory is getting trashed somewhere or something... No idea.

Unfortunately I forgot to release version 0.12.10 which was stuffing up my line numbers compared to your logs. Are you able to update and re-run that just so I can get more accurate line numbers?

u3shit commented 4 years ago

Here's the output recompiled with 0.12.10 from the dev branch (42ee155)

$ ./simple_playback /mnt/ram/A\ Journey\ Awaits.flac 
Press Enter to quit...../../extras/dr_flac.h:8850:48: runtime error: left shift of negative value -104
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8850:48 in 
../../extras/dr_flac.h:8851:48: runtime error: left shift of negative value -4428
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8851:48 in 
../../extras/dr_flac.h:9111:51: runtime error: left shift of negative value -406
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9111:51 in 
../../extras/dr_flac.h:9112:51: runtime error: left shift of negative value -22
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9112:51 in 
../../extras/dr_flac.h:9116:51: runtime error: left shift of negative value -834
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9116:51 in 
../../extras/dr_flac.h:9117:51: runtime error: left shift of negative value -790
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9117:51 in 
../../extras/dr_flac.h:9290:52: runtime error: left shift of negative value -22
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9290:52 in 
../../extras/dr_flac.h:9291:52: runtime error: left shift of negative value -58
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9291:52 in 
../../extras/dr_flac.h:8692:48: runtime error: left shift of negative value -2861
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8692:48 in 
../../extras/dr_flac.h:8693:48: runtime error: left shift of negative value -1280
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8693:48 in 

And with no simd:

$ ./simple_playback /mnt/ram/A\ Journey\ Awaits.flac
Press Enter to quit...../../extras/dr_flac.h:8797:53: runtime error: left shift of negative value -2
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8797:53 in
../../extras/dr_flac.h:8798:53: runtime error: left shift of negative value -2
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8798:53 in
../../extras/dr_flac.h:8795:53: runtime error: left shift of negative value -5
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8795:53 in
../../extras/dr_flac.h:8796:53: runtime error: left shift of negative value -3
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8796:53 in
../../extras/dr_flac.h:8792:53: runtime error: left shift of negative value -1
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8792:53 in
../../extras/dr_flac.h:8793:53: runtime error: left shift of negative value -3
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8793:53 in
../../extras/dr_flac.h:8790:53: runtime error: left shift of negative value -4
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8790:53 in
../../extras/dr_flac.h:8791:53: runtime error: left shift of negative value -2
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8791:53 in
../../extras/dr_flac.h:8816:48: runtime error: left shift of negative value -104
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8816:48 in
../../extras/dr_flac.h:8817:48: runtime error: left shift of negative value -4428
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8817:48 in
../../extras/dr_flac.h:8959:56: runtime error: left shift of negative value -1
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8959:56 in
../../extras/dr_flac.h:8960:56: runtime error: left shift of negative value -54
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8960:56 in
../../extras/dr_flac.h:8961:56: runtime error: left shift of negative value -83
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8961:56 in
../../extras/dr_flac.h:8975:38: runtime error: left shift of negative value -98
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8975:38 in
../../extras/dr_flac.h:8976:38: runtime error: left shift of negative value -154
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8976:38 in
../../extras/dr_flac.h:8979:38: runtime error: left shift of negative value -14
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8979:38 in
../../extras/dr_flac.h:8980:38: runtime error: left shift of negative value -118
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8980:38 in
../../extras/dr_flac.h:8981:38: runtime error: left shift of negative value -176
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8981:38 in
../../extras/dr_flac.h:8958:56: runtime error: left shift of negative value -217
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8958:56 in
../../extras/dr_flac.h:8973:38: runtime error: left shift of negative value -422
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8973:38 in
../../extras/dr_flac.h:8974:38: runtime error: left shift of negative value -688
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8974:38 in
../../extras/dr_flac.h:8978:38: runtime error: left shift of negative value -444
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8978:38 in
../../extras/dr_flac.h:8966:56: runtime error: left shift of negative value -2
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8966:56 in
../../extras/dr_flac.h:8964:56: runtime error: left shift of negative value -2
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8964:56 in
../../extras/dr_flac.h:8965:56: runtime error: left shift of negative value -2
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8965:56 in
../../extras/dr_flac.h:8963:56: runtime error: left shift of negative value -6
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8963:56 in
../../extras/dr_flac.h:9040:47: runtime error: left shift of negative value -406
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9040:47 in
../../extras/dr_flac.h:9041:47: runtime error: left shift of negative value -22
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9041:47 in
../../extras/dr_flac.h:9045:53: runtime error: left shift of negative value -417
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9045:53 in
../../extras/dr_flac.h:9046:53: runtime error: left shift of negative value -395
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9046:53 in
../../extras/dr_flac.h:9254:53: runtime error: left shift of negative value -2
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9254:53 in
../../extras/dr_flac.h:9253:53: runtime error: left shift of negative value -3
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9253:53 in
../../extras/dr_flac.h:9251:53: runtime error: left shift of negative value -8
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9251:53 in
../../extras/dr_flac.h:9252:53: runtime error: left shift of negative value -7
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9252:53 in
../../extras/dr_flac.h:9247:53: runtime error: left shift of negative value -2
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9247:53 in
../../extras/dr_flac.h:9248:53: runtime error: left shift of negative value -4
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9248:53 in
../../extras/dr_flac.h:9249:53: runtime error: left shift of negative value -4
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9249:53 in
../../extras/dr_flac.h:9246:53: runtime error: left shift of negative value -6
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9246:53 in
../../extras/dr_flac.h:9267:52: runtime error: left shift of negative value -22
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9267:52 in
../../extras/dr_flac.h:9268:52: runtime error: left shift of negative value -58
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9268:52 in
../../extras/dr_flac.h:8634:52: runtime error: left shift of negative value -393
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8634:52 in
../../extras/dr_flac.h:8635:52: runtime error: left shift of negative value -248
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8635:52 in
../../extras/dr_flac.h:8632:52: runtime error: left shift of negative value -387
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8632:52 in
../../extras/dr_flac.h:8633:52: runtime error: left shift of negative value -287
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8633:52 in
../../extras/dr_flac.h:8638:52: runtime error: left shift of negative value -139
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8638:52 in
../../extras/dr_flac.h:8639:52: runtime error: left shift of negative value -518
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8639:52 in
../../extras/dr_flac.h:8640:52: runtime error: left shift of negative value -380
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8640:52 in
../../extras/dr_flac.h:8637:52: runtime error: left shift of negative value -442
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8637:52 in
../../extras/dr_flac.h:8658:39: runtime error: left shift of negative value -2861
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8658:39 in
../../extras/dr_flac.h:8659:39: runtime error: left shift of negative value -1280
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8659:39 in

Playback itself seem fine, as I mentioned above I think this is defined behaviour in msvc anc gcc (and by extension clang), and probably with any compiler targeting 2's complement notation... Oh, and also note that the error message can be a bit misleading, the left hand side of the << is negative, not the right, so we have -4428 << 16 and the like, and NOT 1 << -4428.

mackron commented 4 years ago

Thanks for the clarification there. I indeed read that as the right side of the shift. I will look into these errors and get a fix out. Thanks!

mackron commented 4 years ago

OK, I've pushed a potential fix for these errors to the dev branch. Could you give that a go for me and let me know how it goes? I'm expecting there may be a few lingering ones, in particular with stereo mid-side decoding, but let's see how this change works so far.

The stupid thing is I knew this behaviour was undefined, but it didn't click that the message was talking about the left side and not the right side...

Commit: https://github.com/mackron/dr_libs/commit/c8f556565f6b3d453e2f9fb5a37284ffcc54fdd0

kcgen commented 4 years ago

Big improvement! This is what's left from my side:

dr_flac.h:10013:66: runtime error: left shift of negative value -36
dr_flac.h:10014:66: runtime error: left shift of negative value -24
u3shit commented 4 years ago
$ ./simple_playback /mnt/ram/A\ Journey\ Awaits.flac 
Press Enter to quit...../../extras/dr_flac.h:9116:51: runtime error: left shift of negative value -834
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9116:51 in 
../../extras/dr_flac.h:9117:51: runtime error: left shift of negative value -790
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9117:51 in 

And with -DDR_FLAC_NO_SIMD:

$ ./simple_playback /mnt/ram/A\ Journey\ Awaits.flac 
Press Enter to quit...../../extras/dr_flac.h:8975:38: runtime error: left shift of negative value -98
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8975:38 in 
../../extras/dr_flac.h:8976:38: runtime error: left shift of negative value -154
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8976:38 in 
../../extras/dr_flac.h:8979:38: runtime error: left shift of negative value -14
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8979:38 in 
../../extras/dr_flac.h:8980:38: runtime error: left shift of negative value -118
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8980:38 in 
../../extras/dr_flac.h:8981:38: runtime error: left shift of negative value -176
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8981:38 in 
../../extras/dr_flac.h:8973:38: runtime error: left shift of negative value -422
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8973:38 in 
../../extras/dr_flac.h:8974:38: runtime error: left shift of negative value -688
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8974:38 in 
../../extras/dr_flac.h:8978:38: runtime error: left shift of negative value -444
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:8978:38 in 
../../extras/dr_flac.h:9045:53: runtime error: left shift of negative value -417
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9045:53 in 
../../extras/dr_flac.h:9046:53: runtime error: left shift of negative value -395
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:9046:53 in
kcgen commented 4 years ago

I've only tested 44.1 KHz / 16-bit / Stereo; maybe other frequencies, bit-sizes, and channel-counts will find more?

mackron commented 4 years ago

Thanks for running that again. I've pushed another update which I think should address all of those errors in your most recent lists.

Commit: 55055f7800e9040f1483b99dac6f7b35965a2910

kcgen commented 4 years ago

Running clean on my side!

u3shit commented 4 years ago

The normal version looks fine, but I still get errors from the no SIMD version:

../../extras/dr_flac.h:10739:38: runtime error: left shift of negative value -98
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:10739:38 in 
../../extras/dr_flac.h:10740:38: runtime error: left shift of negative value -154
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:10740:38 in 
../../extras/dr_flac.h:10743:38: runtime error: left shift of negative value -14
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:10743:38 in 
../../extras/dr_flac.h:10744:38: runtime error: left shift of negative value -118
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:10744:38 in 
../../extras/dr_flac.h:10745:38: runtime error: left shift of negative value -176
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:10745:38 in 
../../extras/dr_flac.h:10737:38: runtime error: left shift of negative value -422
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:10737:38 in 
../../extras/dr_flac.h:10738:38: runtime error: left shift of negative value -688
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:10738:38 in 
../../extras/dr_flac.h:10742:38: runtime error: left shift of negative value -444
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:10742:38 in 
../../extras/dr_flac.h:10809:62: runtime error: left shift of negative value -417
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:10809:62 in 
../../extras/dr_flac.h:10810:62: runtime error: left shift of negative value -395
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:10810:62 in 

Also, I'm not sure whether I should report it here or open a new issue, but with testing some other random flac files I got some signed overwlows:

../../extras/dr_flac.h:10381:37: runtime error: signed integer overflow: 176952320 - -2109508096 cannot be represented in type 'int'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:10381:37 in 
../../extras/dr_flac.h:10382:37: runtime error: signed integer overflow: 196718848 - -2033911040 cannot be represented in type 'int'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:10382:37 in 
../../extras/dr_flac.h:10380:37: runtime error: signed integer overflow: -1229785088 - 2049710080 cannot be represented in type 'int'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:10380:37 in 
../../extras/dr_flac.h:10383:37: runtime error: signed integer overflow: 889192448 - -1338769408 cannot be represented in type 'int'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:10383:37 in 
../../extras/dr_flac.h:10398:35: runtime error: signed integer overflow: -1010171904 - 1510998016 cannot be represented in type 'int'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:10398:35 in 
../../extras/dr_flac.h:10551:37: runtime error: signed integer overflow: 715259904 + 2141716480 cannot be represented in type 'int'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:10551:37 in 
../../extras/dr_flac.h:10552:37: runtime error: signed integer overflow: 740818944 + 2082734080 cannot be represented in type 'int'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:10552:37 in 
../../extras/dr_flac.h:10553:37: runtime error: signed integer overflow: 750321664 + 2048720896 cannot be represented in type 'int'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:10553:37 in 
../../extras/dr_flac.h:10554:37: runtime error: signed integer overflow: 749928448 + 2026504192 cannot be represented in type 'int'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../extras/dr_flac.h:10554:37 in 
mackron commented 4 years ago

The shifting warnings should now be fixed: https://github.com/mackron/dr_libs/commit/787651cff77aa2f1df2d7eb6d118e38609d9ea71

I have half a mind to just not fix the overflow warnings. It's coming from the code below, and all of this casting just to get rid of these warnings is getting to be beyond ridiculous and I don't want to be maintaining this crap. Do you have an idea to make this clean? Could try making left and side unsigned and then cast to signed as the last step, maybe?

drflac_int32 left0 = (drflac_int32)((drflac_uint32)pInputSamples0[i*4+0] << shift0);
drflac_int32 left1 = (drflac_int32)((drflac_uint32)pInputSamples0[i*4+1] << shift0);
drflac_int32 left2 = (drflac_int32)((drflac_uint32)pInputSamples0[i*4+2] << shift0);
drflac_int32 left3 = (drflac_int32)((drflac_uint32)pInputSamples0[i*4+3] << shift0);

drflac_int32 side0 = (drflac_int32)((drflac_uint32)pInputSamples1[i*4+0] << shift1);
drflac_int32 side1 = (drflac_int32)((drflac_uint32)pInputSamples1[i*4+1] << shift1);
drflac_int32 side2 = (drflac_int32)((drflac_uint32)pInputSamples1[i*4+2] << shift1);
drflac_int32 side3 = (drflac_int32)((drflac_uint32)pInputSamples1[i*4+3] << shift1);

drflac_int32 right0 = left0 - side0; // <-- Overflow warnings here.
drflac_int32 right1 = left1 - side1;
drflac_int32 right2 = left2 - side2;
drflac_int32 right3 = left3 - side3;
kcgen commented 4 years ago

Yeah; casting hurts readability. I'm not sure if it's possible to clamp values to a valid range, so the subsequent math still produces sane (or non-overflowing) numbers?

mackron commented 4 years ago

No, that won't work unfortunately. I'm testing with doing everything as unsigned and then casting to signed at the very last step. Seems to be working so far, but will need some testing. Will push an update in a few hours.

kcgen commented 4 years ago

Very nice - maybe a triple-win as well!

mackron commented 4 years ago

OK, I've pushed a change to use unsigned integers for everything (still requires some casting, but better than it was): https://github.com/mackron/dr_libs/commit/819484686395c2fc90f657c84f17d2d7f64033c2

kcgen commented 4 years ago

Zero runtime errors and proper playback confirmed with all permutations of {mono or stereo} channels having {8, 16, or 24} bit samples, at {11025, 22050, 44100, 48000, 88200, or 96000} Hz. Testing with gcc 9.3.0 UASAN on x86-64 Ubuntu 20.04.

u3shit commented 4 years ago

Thanks, it looks much better now, I haven't received any errors on x64 (with or without SSE). However in case I'm not annoying enough, I've decided to take my rpi2 to a spin and look into the neon specific code (hey, it's only takes 1.5 min to compile the simple_playback example, without optimization!). This is what I got with gcc 9.2:

/usr/lib/gcc/armv7a-unknown-linux-gnueabihf/9.2.0/include/arm_neon.h:1929:14: runtime error: signed integer overflow: 176952320 - -2109508096 cannot be represented in type 'int'
/usr/lib/gcc/armv7a-unknown-linux-gnueabihf/9.2.0/include/arm_neon.h:598:14: runtime error: signed integer overflow: -101928960 + -2145284864 cannot be represented in type 'int'

Which is pretty useless without a stacktrace... But line 1929 is vsubq_s32 and 598 is vaddq_s32.

u3shit commented 4 years ago

I've managed to get a stacktrace for the first one, unfortunately I can't really filter out identical stacktraces if I just break on __ubsan_handle*, so that's what I can provide for now.

#0  0x00455890 in __ubsan_handle_sub_overflow@plt ()
#1  0x00513bb4 in vsubq_s32 (__b=..., __a=...) at /usr/lib/gcc/armv7a-unknown-linux-gnueabihf/9.2.0/include/arm_neon.h:1929
#2  drflac_read_pcm_frames_s32__decode_left_side__neon (pOutputSamples=0x7632bddc, pInputSamples1=0x6ec100, pInputSamples0=0x6e8100, unusedBitsPerSample=8, frameCount=448, pFlac=0x6e6208) at ../../extras/dr_flac.h:8725
#3  drflac_read_pcm_frames_s32__decode_left_side (pOutputSamples=0x7632bddc, pInputSamples1=0x6ec100, pInputSamples0=0x6e8100, unusedBitsPerSample=8, frameCount=448, pFlac=0x6e6208) at ../../extras/dr_flac.h:8749
#4  drflac_read_pcm_frames_s32 (pFlac=0x6e6208, framesToRead=448, pBufferOut=0x7632bddc) at ../../extras/dr_flac.h:9382
#5  0x0063f2c8 in ma_decoder_internal_on_read_pcm_frames__flac (pDecoder=0x7eff73f0, pFramesOut=0x7632bddc, frameCount=448) at ../../miniaudio.h:39633
#6  0x006460b4 in ma_decoder_read_pcm_frames (pDecoder=0x7eff73f0, pFramesOut=0x7632bddc, frameCount=448) at ../../miniaudio.h:41131
#7  0x00656ed4 in data_callback (pDevice=0x7eff9c00, pOutput=0x7632bddc, pInput=0x0, frameCount=448) at ../simple_playback.c:23
#8  0x005abdfc in ma_device__on_data (pDevice=0x7eff9c00, pFramesOut=0x7632bddc, pFramesIn=0x0, frameCount=448) at ../../miniaudio.h:9001
#9  0x005ac438 in ma_device__read_frames_from_client (pDevice=0x7eff9c00, frameCount=448, pFramesOut=0x7632bddc) at ../../miniaudio.h:9029
#10 0x005c1bc4 in ma_device_main_loop__alsa (pDevice=0x7eff9c00) at ../../miniaudio.h:18142
#11 0x005e5fe4 in ma_worker_thread (pData=0x7eff9c00) at ../../miniaudio.h:29561
#12 0x76f14b7c in start_thread () from /lib/libpthread.so.0
#13 0x769ce8e8 in ?? () from /lib/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
mackron commented 4 years ago

I'll have to dig out my Raspberry Pi and take a look. I must say, I'm surprised this would be reported as undefined behaviour considering it's an architecture specific instruction which I would have thought would be well defined in this regard.

u3shit commented 4 years ago

If you look at the header, vsubq_s32 is just a plain inline function with some weird types and a return __a - __b; body, it doesn't call any __builtin functions like some other intrinsics functions in this header. Since this is a normal subtract, it makes sense it warns, whether this is normal in case of this function, I don't know. Clang's arm_neon.h is similar as it simply substracts, but looks like in the past gcc used a builtin: https://android.googlesource.com/platform/prebuilts/gcc/darwin-x86/arm/arm-eabi-4.7/+/3067d9830af26242f3e2ea309f46484630b0b8e2/lib/gcc/arm-eabi/4.7/include/arm_neon.h

mackron commented 4 years ago

I've gone ahead and pushed a potential fix for the NEON path. Could you let me know if that fixes it for you?

Commit: https://github.com/mackron/dr_libs/commit/5cac59531b29458a3ca4ec154c30d4a1480d3854

u3shit commented 4 years ago

@mackron Thanks, it works without warnings with gcc 9.2.0 on my raspberry pi 2 with the samples I checked. Maybe if there's some kind of flac sample repository I could check them.

mackron commented 4 years ago

I'm not aware of any such repositories, unfortunately. I've used OC Remix in the past to acquire some realistic FLAC music files, though.

I will go ahead and release these changes and close this issue. If you stumble across any more of these errors feel free to open a new issue.

Thanks for the report and testing!