Extra sounds while playing MIDI music on Xubuntu 20.04

[lua-rocks]

Your question

When I start the game, I hear two tunes, one on top of the other - the usual background midi music and some awful pipes on top. Is it possible to turn off these nightmarish pipes? Thank you.

Additional info

No response

[ihhub]

Hi @lua-rocks , could you please provide more information about your issue? It's not a question but rather a bug. At minimum we need your operating system information, which exact version of fheroes2 you're running, your hardware. What version of Homm2 do you use? Does external music work?

[lua-rocks]

Hi @ihhub , here is my system info:

                          ./+o+-       luarocks@MAINFRAME
                  yyyyy- -yyyyyy+      OS: Xubuntu 20.04 focal
               ://+//////-yyyyyyo      Kernel: x86_64 Linux 5.4.0-84-generic
           .++ .:/++++++/-.+sss/`      Uptime: 7h 50m
         .:++o:  /++++++++/:--:/-      Packages: 2409
        o:+o+:++.`..```.-/oo+++++/     Shell: fish 3.1.0
       .:+o:+o/.          `+sssoo+/    Resolution: 1280x720
  .++/+:+oo+o:`             /sssooo.   DE: Xfce
 /+++//+:`oo+o               /::--:.   WM: Xfwm4
 \+/+o+++`o++o               ++////.   WM Theme: Greybird
  .++.o+++oo+:`             /dddhhh.   GTK Theme: Greybird [GTK2]
       .+.o+oo:.          `oddhhhh+    Icon Theme: elementary-xfce-darker
        \+.++o+o``-````.:ohdhhhhh+     Font: Noto Sans 11
         `:o+++ `ohhhhhhhhyo++os:      Disk: 141G / 456G (33%)
           .o:`.syhhhhhhh/.oo++o`      CPU: AMD Phenom II X4 945 @ 4x 3GHz
               /osyyyyyyo++ooo+++/     GPU: GeForce GTX 465
                   ````` +oo+++o\:     RAM: 910MiB / 3935MiB
                          `oo++. 

fheroes2 version: 0.9.7 (05 September 2021)

I hear these pipes using any files - from demo or from original homm2 sucession wars.

Also I've tried to switch option music in ~/.config/fheroes2/fheroes2.cfg to all possible values (original, expansion, external) and it doesn't help.

[ihhub]

@lua-rocks , what version of SDL do you use? Did you compile by yourself? If not can you try to do? I think some users experiences sound issues in the past due to incorrect libraries/drivers installed at the system.

[lua-rocks]

I have not compiled anything by myself, but for the sake of experimentation I can. I have installed all the packages beginning with libsdl from the Ubuntu 20.04 repositories (versions 1 and 2 - 64 and 32 bit).

[ihhub]

We provide SDL 1 compiled version and SDL 2 compiled version for Ubuntu OS. SDL2 is preferred.

[lua-rocks]

I had a fheroes2_ubuntu_sdl2.zip and I just downloaded fheroes2_ubuntu_sdl1.zip, there is no difference between them. I hear pipes mixed with midi in both versions in main menu.

[lua-rocks]

Maybe its possible to just remove the files with non-midi music? Unfortunately I don't know where they are located.

[ihhub]

Midi files are the part of the game resources. Which version of HoMM2 do you use? You can at least reduce the volume of music to 0.

[lua-rocks]

I hear pipes in any version, even in the free demo. I know that midi music is a game resource, but these pipes are not midi music and I don't want to turn off midi.

[ihhub]

Let's wait for other developers to find the root of the problem. I don't have installed Xubuntu version to verify it as before nobody had such issue. As I mentioned it might be something to do with MIDI package or drivers for sound.

[lua-rocks]

Listen:

Here is original midi recorded from dosbox (and I love it so much! 💖):

https://user-images.githubusercontent.com/82697845/133956523-01cba90f-8bbb-4df7-8823-655af7fcdf9d.mp4

And here is what I hear from fheroes (sounds like a saw 🥺):

https://user-images.githubusercontent.com/82697845/133956597-28223483-43c8-4d88-8e71-141d14c5311d.mp4

[lua-rocks]

At the game map I hear only midi, but in castles I hear 2 mixed musics with different volume.

[oleg-derevenetz]

Hi @lua-rocks something is definitely wrong with MIDI sound on your system. That's how main menu theme in MIDI mode sounds in the original game (HoMM2 Gold from GOG, DOSBOX) on my Windows system:

https://user-images.githubusercontent.com/32623900/133984697-66adbb99-3f3f-443c-8b84-54d93d73d97e.mp4

And it definitely sounds similar here, for example:

https://www.youtube.com/watch?v=bYZSfGWedGQ

And it sounds in the very same way in fheroes2 too - both on my Windows and macOS machines (in MIDI mode). So there definitely should be pipes in background, but they should sound in a different way. And for some reason they do not sound at all in your DOSBOX.

[lua-rocks]

In this case, I confirm that it is a bug. Thank you for your example, @oleg-derevenetz , it sounds correct. Not like mine at all.

[lua-rocks]

I'll try now to remove all packages with "midi" and "libsdl" in their names and install only the necessary ones from scratch.

[lua-rocks]

Very strange, but it didn't help! 😢

[lua-rocks]

Along with libsdl/midi I uninstalled a huge amount of depended software, and all for nothing.

[lua-rocks]

Can anyone confirm that this bug is valid for any OS based on Ubuntu Focal (latest LTS), not just mine?

[lua-rocks]

I just built the game from sources and that also has not solved the problem. 😭

[lua-rocks]

Now I know for sure it's a bug, because it's not in version 0.7 and it appears for the first time in version 0.8!

[lua-rocks]

Sorry for spam :D

[oleg-derevenetz]

@lua-rocks what is interesting, sound in your DOSBOX version of the original game is wrong too: main menu theme should sound different, like in my video or that YouTube video. In case of your DOSBOX, not all instruments are playing.

[lua-rocks]

I think there all instrument are playing but they have a different sound. It's ok for midi format. Midi is not mp3 :) In any case, it is not related to my issue because the sound I hear in fheroes has not just a different instruments, but a completely different tune.

[felix642]

Well, I can confirm this bugs seems to be happening on linux. I have the same issue on my machine.

https://user-images.githubusercontent.com/60141446/134438346-06981142-76a9-49d3-97bc-bff7be91181d.mp4

[undef21]

@lua-rocks and @felix642 , could you please check if package freepats is installed and install this if no.

[lua-rocks]

@undef21 it was not installed and after installing it nothing has changed.

[lua-rocks]

Here is quick example of castles sounds. They are set to midi but this is not (only) a midi:

https://user-images.githubusercontent.com/82697845/134755406-86dc54e9-a87b-4755-ae18-c43e415f1c51.mp4

[ihhub]

Hi @lua-rocks , as far as I understood the problem lies with your system setup / drivers, not with fheroes2 engine itself. If it's true we can proceed with this issue closure as it's beyond the scope of this project.

[lua-rocks]

@ihhub the problem lies with any Ubuntu-based Linux distribution, including Ubuntu itself.

[lua-rocks]

@ihhub My configuration is not special, it's out of the box behavior and this bug is not present in versions earlier than 0.8.

[lua-rocks]

@ihhub please check changes in sound between 0.7 and 0.8. The problem lies there.

[ihhub]

Hi @idshibanov , could you please take a look at this issue as you were the one changing this part?

[oleg-derevenetz]

It would be great to perform a git bisect to find the exact commit after which the issue began to be observed. Unfortunately, this issue is not reproducible on any of my devices, so maybe someone who observed this issue may help.

[lua-rocks]

On my potato-pc it will take a years, but I'll try!

[lua-rocks]

... Ok, maybe not a years :D

@oleg-derevenetz here is it: [e77f41b6280b4f71ca8faf3a2b731fd2dcf46c5b] Music: xmi2mid rework (#521)

[oleg-derevenetz]

Hi @lua-rocks I just tried to make a variant with 3rd party implementation of xmi2mid converter, you can clone my fork (https://github.com/oleg-derevenetz/fheroes2) and switch to the xmi2mid branch. I wonder, will there be any difference on your side (for me, it sounds pretty similar to the current fheroes2 music)? This is just a test branch to find the issue, so it is not 100% reliable (there may be memory leaks and so on), but it should be enough to test MIDI music in main menu and a few castles.

[lua-rocks]

@oleg-derevenetz thank you! I built your branch, it has no sound at all and the game crashes very quickly, spamming in terminal with messages "fluidsynth: warning: Ringbuffer full, try increasing polyphony!".

[oleg-derevenetz]

@lua-rocks Interesting. There is definitely some polyphony issue in your setup, and DOSBOX deals with this somehow (as far as I know, DOSBOX has some sort of FluidSynth integration, so it may be aware of this) - apparently by limiting the number of instruments (as I already mentioned before, definitely not all instruments are playing in your DOSBOX, there should be more) while with SDL there are issues.

[lua-rocks]

If you find free time, please install xubuntu, lubuntu or another lightweight ubuntu-based distribution in virtualbox and check it yourself (although the emulation of the sound driver may be different from the real machine, I am not sure about this).

[oleg-derevenetz]

although the emulation of the sound driver may be different from the real machine, I am not sure about this

That's the problem. I already have Ubuntu running in VM on my macOS and there is no issue.

[undef21]

It's interesting. There is no this issue on Debian 9, but Ubuntu 20.04 in VBox has it.

[Gerwin2k]

I just encountered this issue as well, on the same Windows XP system (Yeah I know, just messing around):

1) Official v0.9.8 package "fheroes2_windows_x86_SDL2.zip": Menu midi music channel 3 is set to "Church Organ". This sounds correct.

2) My WIN32 build of current SVN: Menu midi music channel 3 is set to "Strings". This is the wrong instrument.

I can see this by checking the status of the software Midi synthesizer. It is the same every time. Most of the midi track is running fine, but maybe some data at the very start of the midi stream is not send out reliably?

Edit: I picked an older source package "fheroes2-fheroes2-linux-sdl2_dev.zip" from 10-2019 and took the "xmi2mid.cpp" file from there (with minor adjustments to make it compile). Resulting WIN32 build now has the proper midi instrument at the main menu.

[oleg-derevenetz]

Hi @Gerwin2k

I added some debugging to the MidiEvents constructor from the xmi2mid.cpp:

--- a/src/engine/xmi2mid.cpp
+++ b/src/engine/xmi2mid.cpp
@@ -392,6 +392,7 @@ struct MidiEvents : std::vector<MidiChunk>

                     // program change
                     case 0x0C:
+                        printf("PROGRAM: %d %d\n", (int)(*ptr & 0x0F), (int)*(ptr + 1));
                     // channel aftertouch
                     case 0x0D:
                         emplace_back( delta, *ptr, *( ptr + 1 ) );

and here is the list of PROGRAM CHANGE (0xC0) commands in MIDI and MIDI Expansion modes for main menu theme (the first number is MIDI channel number, and the second is program number - that is, a musical instrument):

PROGRAM: 1 49 (String Ensemble 1)
PROGRAM: 2 48 (Timpani)
PROGRAM: 4 73 (Piccolo)
PROGRAM: 3 6 (Chorused Piano)
PROGRAM: 6 60 (Trumpet)
PROGRAM: 2 19 (Rock Organ)

That's the list of instruments used by original XMI track. However, I suppose, not all instruments may be supported by a particular sound driver, so may be some instrument remapping may take place.

[Gerwin2k]

Hello Oleg. Thanks for checking.

However, I suppose, not all instruments may be supported by a particular sound driver, so may be some instrument remapping may take place.

This is ancient midi data, probably 'standard GM' without any Roland GS extensions. There is not much of a driver involved, except a midi port that throws every bit of data directly to the software or hardware synthesizer at the other end. In the above test I used Sound Canvas software synthesizer. The 'industry standard'. It has a GUI that shows what it is doing, it shows the program data that it is receiving from the windows software midi port. https://static.roland.com/assets/images/products/main/rc_sound_canvas_va_main.jpg

Swapping the "xmi2mid.cpp" file version made the difference in my test. Both my tests, with good and bad instrument, were done on the same system, under the same circumstances. When I have time I will try to find which part of file "xmi2mid.cpp" code made the difference.

[Gerwin2k]

I tested different versions of xmi2mid, and this is the 'guilty' commit that introduces the mentioned 'Strings' instrument instead of 'Church Organ', with my WIN32 compiles: e77f41b6280b4f71ca8faf3a2b731fd2dcf46c5b of 2020-05-16. As usual this commit is extensive and I have no idea yet what part of the commit is responsible.

As already noted: occurrence of the bug seems to depend on the used compiler and/or system. Since the official WIN32 distribution does not have this behavior on my system.

[oleg-derevenetz]

I even exported resulting MIDI to the file:

menu.zip

using the following patch:

diff --git a/src/engine/xmi2mid.cpp b/src/engine/xmi2mid.cpp
index 5e376f83..2cc1f8af 100644
--- a/src/engine/xmi2mid.cpp
+++ b/src/engine/xmi2mid.cpp
@@ -525,5 +525,11 @@ std::vector<uint8_t> Music::Xmi2Mid( const std::vector<uint8_t> & buf )
         sb << mid;
     }

+    static int i = 0;
+
+    std::ofstream ofs(std::to_string(i++) + ".mid", std::ios::binary);
+
+    ofs.write(reinterpret_cast<const char *>(sb.data()), sb.size());
+
     return std::vector<u8>( sb.data(), sb.data() + sb.size() );
 }

Honestly, I don't see any problems with this file. I tried to play it with VLC Player and GarageBand, and it sounds a bit different in each player and in fheroes2 itself, so I suppose that some sort of instrument remapping takes place (or each player has its own library of instruments that sound a bit different), but in general LGTM. I don't think that there is any issue in xmi2mid. xmi2mid does not work with instruments at all, it passes the PROGRAM CHANGE commands through from the XMI as is.

[Gerwin2k]

I suspect you cannot reproduce the issue, as it seems MSVC compiles don't have it. I use GCC 8 here. Linux builds are also GCC.

Meanwhile I was messing with xmi2mid.cpp, and this stable_ variant of the sort function seems to fix things, present on line 411:

//std::sort( this->begin(), this->end() );
std::stable_sort( this->begin(), this->end() );

[oleg-derevenetz]

I suspect you cannot reproduce the issue, as it seems MSVC compiles don't have it. I use GCC 8 here. Linux builds are also GCC.

I use clang from Xcode on macOS.

Meanwhile I was messing with xmi2mid.cpp, and this stable_ variant of the sort function seems to fix things, present on line 411:

Aha, interesting. So currently MIDI commands may be reordered if they have the same timestamp, and replacing sort with stable_sort disables reordering. Good catch, I'm not very familiar with MIDI format, but it looks like reordering may break things. Do you want to propose a pull request?

[oleg-derevenetz]

@lua-rocks @undef21 Can you please verify that replacing std::sort with std::stable_sort fixes the issue in your case?

[Gerwin2k]

Ah I did not even understand that logic yet. I was just fairly certain those six lines of re-ordering at 411 were likely the cause, as commenting them out gave garbled music but proper instruments. Also it had to be a compiler difference, since the official MSVC WIN32 build does not have the issue on my system.

If you can commit the change that would be great. It is fixed for my own use, so I am done. :)