Shoes Sound API (formerly Bloopsaphone)

[IanTrudel]

Shoes provided a Sound API at some point through Bloopsaphone extension. Bloosaphone is based on cross-platform library PortAudio. @ccoupe reports version 2.0 caused his audio system to break on Linux. He further suggests to detect whether PortAudio is installed or not, implement it through FFI (i.e. VLC) rather than integrate it to Shoes,

Latest stable version of PortAudio is 19.6.0. Notes about building PortAudio with MinGW

Additional references:

• Blu Tuner is a Bloopsaphone player
• OpenAL is an alternative sound library
• LAME encode and decode MP3

[ccoupe]

What would be really good is if someone (else) made a gem for bloopsaphone - with precompiled Windows binaries inside so Shoes doesn't depend on it. Said gem might work for some, not for others and it's not a Shoes problem.

[ccoupe]

3 gems:

bloopsaphone (0.4)
viking-bloopsaphone (0.4.1)
why-bloopsaphone (0.4)

curious name on the third one.

[IanTrudel]

There are also those 2-5 years outdated projects:

• Ruby-PortAudio
• Easy Audio
• FFI PortAudio

OpenAL seems a bit more recent.

[ccoupe]

Tells us something about portaudio. Dead? or don't touch?

[IanTrudel]

Tells us something about portaudio. Dead? or don't touch?

Neither. Even Audacity uses PortAudio. This is more a generalized problem in Ruby where libraries get abandoned, replaced, etc.

A bit confused why you want to push Bloopsaphone as far away from Shoes as possible. So many other things were implemented within Shoes rather than ext and most certainly not gems. This is a low priority issue anyway.

[ccoupe]

Ext's and Gems with binaries that are included in Shoes have to be built and someone has to put the stake in the ground - "I support this version in Shoes and I'll keep it running". Is that me? If I could I wouldn't ship any ext or binary gems and every one install the tools needed to build any gem they run across. But that's not backwards compatible. I try to minimize my effort building and testing new exts or binary gem on 5 platforms. Let me repeat, Five builds and Five tests and Shoes tests mean running the damn thing => Install new version of Shoes with the gem. clicking on whatever it takes to get the sample/bug/test script running. Repeat many times. (x5)

portaudio is not included in raspbian for example, so I would have to install it, copy it and include that .so in that Shoes and hope it doesn't screw up whatever sound system a raspbian user is using.

Or some clever person could write some ffi to work with whatever the current host offers for sound api , gem-itize it and Shoes users can just 'require' it.

[IanTrudel]

We certainly both agree that Shoes should be easier to maintain. My understanding from the Bloopsaphone code is that it shouldn't be difficult to make it work. That I can give a try when Msys2 target is ready. It would then be easier to decide whether to reintegrate or remove it from Shoes.

portaudio is not included in raspbian for example, so I would have to install it, copy it and include that .so in that Shoes and hope it doesn't screw up whatever sound system a raspbian user is using.

I suspect the version of PortAudio that screw up your system was actually a beta version and/or ALSA may have not been configured properly. Stable release should be fine.

Or some clever person could write some ffi to work with whatever the current host offers for sound api , gem-itize it and Shoes users can just 'require' it.

Do you mean FFI to use PortAudio? That may work. Not sure what "whatever the current host offers for sound api" means but every platform has its own sound API (DirectSound, ALSA, MacOS X Core Audio, etc); that would be a whole lot more work. PortAudio is actually running on top of those APIs and thus cross-platform.

References: PortAudio on Raspberry Pi Advanced Linux Sound Architecture (ALSA

[ccoupe]

Please don't remind me about ALSA (shudder) Instead of waiting for msys2 (not really a gatiing event for this issue) , install a Linux VM and setup a jackd config with dynamic transient usb audio sources and sinks. and then put portaudio on top if it and test all the devices you can afford. I've been deep into te ALSA/pulse-audiot goat roping running a linux real time kernel. I know this ground too well. Too well?

Including portaudio.dll or .so in Shoes means I have to build them and I have to support it every crazy situation that might occur in linux. I am aware of the counter argument that including portaudio/bloopsahpone would make Shoes more attractive to new developers. Speculation or wishful thinking? How to know?

[IanTrudel]

Compiled PortAudio (pa_stable_v190600_20161030.tgz) and then compiled Bloopsaphone from Shoes req directory. This test is working.

I need the new Windows target to make it work with Shoes but it's very likely to work without problem. As mentioned before, Linux version will need flags to be able to make PortAudio coexist with whatever else in the system.

[IanTrudel]

Additional reference (on usage): https://github.com/adminmyserver/ruby_pong

Slightly modified PongSounds is working:

require("bloops")
require 'singleton'

class PongSounds
  include Singleton

  class << self
    def bloops
      @bloops ||= Bloops.new
    end

    def sounds
      @sounds ||= begin
        @hit = bloops.sound Bloops::SQUARE
        @hit.volume = 0.4
        @hit.sustain = 0.02
        @hit.attack = 0.02
        @hit.decay = 0.1
        @hit.punch = 0.1

        @score = bloops.sound Bloops::SAWTOOTH
        @score.volume = 0.2
        @score.sustain = 0.25
        @score.attack = 0.02
        @score.decay = 0.1
        @score.punch = 0.5
        @score.arp = 0.1
        @score.phase = 0.1

        @bounce = bloops.sound Bloops::SINE
        @bounce.volume = 0.5
        @bounce.sustain = 0.1
        @bounce.attack = 0.02
        @bounce.decay = 0.1
        @bounce.punch = 0.5
        @bounce.arp = 0.1
        @bounce.phase = 0.1

        {:hit => [@hit, "32 + A#"], :score => [@score, "16 - D#"], :bounce => [@bounce, "16 - F#"]}
      end

      @sounds
    end

    def play(name)
      Thread.new do
        bloops.tune *sounds[name]
        bloops.play
        sleep 0.1 until bloops.stopped?
        bloops.clear
      end
    end
  end
end

Shoes.app do
   button("play 'em all") do
      [:hit, :score, :bounce].each { |n|
         PongSounds.play(n)
         sleep 1
      }
   end
end

[ccoupe]

portaudio-dev is still evil on ubuntu - I just tried - hopefully I didn't break anything.

[IanTrudel]

What kind of setup and parameters did you compiled it?

[ccoupe]

Standard Ubuntu apt-get settings. Would it be different if my Ubuntu was a couple years newer? Perhaps, perhaps not - no confidence in that project (less after you noitice the code is still marked unstable - no kidding, Sherlock?. Forcing portaudio to do something else is what breaks things, Will not do. It's just as f*cked up as it was so many years ago.

[IanTrudel]

http://portaudio.com/docs/v19-doxydocs/compile_linux.html

[ccoupe]

If you don't understand the linux sound mess,, at the bottom you have devices and device drivers and then a slightly higher level alsa, and mixers, and then there is a gob smacking mess of libraries that pretend to have alsa interfaces some of them incompatible with each unless the end-user goes to great work to wire them properly (pulseaudio, jackd) and then gstreamer , vlc, the terminal bell, has something usable. That is done a at sudo level and very specific to what the user/distro wants.

Portaudio lives at the alsa or psuedo-alsa level. I have PortAudiio 2.0 (bin) installed which is really 1.9+ubuntu-patches. Plays nice with pulseaudio which is what ubuntu uses. (other Linux may use something different. My pi3 has no upper layers for example. So why can't I install portaudio19-dev package to get the headers? Had to use the command-line.

sudo apt-get install portaudio19-dev
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 portaudio19-dev : Depends: libjack-dev

And that if were to install libjack-dev it would drag in jackd and potentially break pulseaudio without a lot of sudo madness. If've run jackd before - great for sound engineers - not for me.

[IanTrudel]

This is very strange. A little lost for words because I never experienced such problems on Linux. There is certainly a solution to the sound issue. Sound always worked for me. Besides, it is challenging to find a cross-platform audio library. Squeak Smalltalk implements sound through platform specific API (DirectSound, ALSA/OSS, OSX Audio, etc.) but we don't have the manpower for such venture.

We have hands full with other tasks at the moment. You are busy with MSYS2 and I am busy refactoring, both more important than sound. Let's keep this issue open and revisit at a later time. What do you say?

[ccoupe]

You never had a jackd installation in Linux ;-). We can't depend on portaudio being installed on linux. Nor can we ship the libportaudio.so with Shoes because it's a 'system level' thing that can break user sound setups. We don't include libvlc.so for similar reasons. I suspect I can trick the bloopsaphone gem into building but it's not going to work on all linux. Raspbian doesn't even have portaudio in their package repo - I wonder if msys2 does. OSX?

[IanTrudel]

Does Audacity work on your Linux?

[ccoupe]

Does Audacity work on your Linux?

Yes, but its using the one of the pseudo ALSA interfaces setup by pulseaudio. Is a portaudio.so involved? Who knows. The problem, once again is I can't install the package with portaudio.h in it without dragging in jackd and reconfiguing my audio settings and that is not going to happen again on my system.

Could I get the .h file from somewhere else and build the portaudo GEM. Probably. Someday.

[IanTrudel]

Just collecting information anyway. If we know something works on your system, it's a bit easier to find a path to make this work.

Yes, but its using the one of the pseudo ALSA interfaces setup by pulseaudio.

This is basically how the Linux version of Audacity works.

The problem, once again is I can't install the package with portaudio.h in it without dragging in jackd and reconfiguing my audio settings and that is not going to happen again on my system.

From http://wiki.audacityteam.org/wiki/Linux_Issues:

Current Audacity supports interfacing with JACK. It can be enabled (or disabled) by configuring PortAudio with JACK and ALSA, although in practice JACK gets enabled whenever the JACK headers are present on the build system.

It should not compile JACK version on your system, unless you have it installed. There should also be a configure switch for it. locate jack.h just in case. My understanding from our conversation so far is that we must avoid JACK at all cost. That is very clear.

Additional references:

• http://manual.audacityteam.org/man/tutorial_recording_computer_playback_on_linux.html

[ccoupe]

BUT I'm using apt-get to install the Ubuntu portaudio-dev - and it was compiled with jackd and has a jackd dependecy when it was built/packaged that apt-get warns about and I know all too well. I could sacrifice a new VM and linux install just to get portaudio built and install ruby 2.2.6 and build the gem but so could anyone else if motivated to spend a day or two on the effort.

[IanTrudel]

For Audacity, --disable-jack according to https://github.com/audacity/audacity/blob/17afc51644b2b327e173a23d6066dde598838c03/lib-src/libsndfile/ChangeLog

For PortAudio, according to ./configure --help

Optional Packages:
  --with-PACKAGE[=ARG]    use PACKAGE [ARG=yes]
  --without-PACKAGE       do not use PACKAGE (same as --with-PACKAGE=no)
  --with-alsa             Enable support for ALSA [autodetect]
  --with-jack             Enable support for JACK [autodetect]
  --with-oss              Enable support for OSS [autodetect]
  --with-asihpi           Enable support for ASIHPI [autodetect]
  --with-winapi           Select Windows API support

That would be a --without-jack.

[ccoupe]

I tried on a Mint 18 VM (unbuntu apt-get based but more current) still drags in jackd and many other things no rational user wants if you let it. In the i386 linux chroot installed alsa, built portaudio from source, and built the bloopsahone gem but the test script complains:

(deb386)ccoupe@bronco:/usr/local/lib/ruby/gems/2.2.0/gems/bloopsaphone-0.4/ext/ruby$ ruby test.rb 
ALSA lib pcm_dmix.c:1018:(snd_pcm_dmix_open) unable to open slave
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
ALSA lib pcm_dmix.c:1018:(snd_pcm_dmix_open) unable to open slave
ALSA lib pcm_dmix.c:1018:(snd_pcm_dmix_open) unable to open slave

It could be the the chroot doesn't allow that. I'm highly unlikely to fix that and it very likely that doing the same in a 86_64 chroot will improve things. But, that doesn't mean the gem won't run given a better Shoes test script (and a real i686 linux with portaudio installed). No, my 32 bit debian VM is too old - so even if it works somewhere in 386 linux, it doesn't work here.

64-bit linux. (Mint) - No Alsa -but the included Vlc shows movies without sound in the vm. VM problem -possible but Windows VM works with the same simulated audio device. It seems bloopsaphone gem is hard wired to use /usr/bin/ruby-2.3 for using the extconf.rb. WTF! 2.3 is no where near or close to this VM.

So many problems to fix: here, there, and for every linux. You can not depend on PortAudio and you can't build it and include it with Shoes/Linux.

[ccoupe]

The bloopsaphone gem needs to test its environment better - it's not using my rvm ruby. Or maybe the error is on me? Without ffi and checking for so/dlll, Shoes can't use PortAudio.

[ccoupe]

Lets set some top level goals for this gem like creature. What ever is included in Shoes will not require installation into system directories or user reconfiguration which might destroy existing sound setups. It can only use what is available. Shoes need to test if Portaudio is available and do nothing if its isn't. The bloopsaphone gem does not do that. Some one has to fix that in the gem and claim ownership of that newer code.

On Windows we could include a libportaudio.dll if someone builds one and the Shoes gem is a lot smarter about it's environment.

OSX is going to be a lot like Linux. Use what you discover.

[IanTrudel]

What do you have in mind? A simple test for the presence of libportaudio.dll?

It would be possible to package a new (unpublished) gem. Not particularly difficult. We could even add some MIDI feature with midi-bloop based on Bloopsaphone and midilib.

[ccoupe]

Yes, it would require a 'Shoes-Only' gem and it could include midi it it can detect the so/dll and adapt it's behavior if the request so/dll is not available.

[ccoupe]

What do you have in mind? A simple test for the presence of libportaudio.dll?

A test for libportaudio.so (linux) and what ever osx uses and someone would have to build portaudio.dll so it can be part of Shoes/windows.

[ccoupe]

Mixed news. I've tried portaudio again. I can build and distributed it and the bloopsaphone gem for Windows. They work. On Linux the user is going to have to install portaudio themselves and in my case install ALSA first (libasound) - Shoes can't and shouldn't fool around in the system libs. OSX is similar to Linux. Ii can be be built (if they have the xcode command line tools) but attempting to include libportaudio.dylib with Shoes tries to drag in system dylibs that should not be part of Shoes.

Someone, not me, needs to step up and maintain the ffi-portaudio gem (or some other gem - the midi one?) and include the Windows libportaudio.dll inside the gem. Or use unimidi. It's all so old and unsupported - not for Shoes inclusion.

[IanTrudel]

This is an encouraging news. It would be ok to bundle PortAudio with Windows and MacOS releases. Linux might need something to check whether the libraries are present. Anyway, good to see some progress.

[ccoupe]

I must have been unclear. PortAudio and OSX won't work for Shoes inclusion. 2 or 3 fails for portaudio and I haven't looked at freebsd and nothing to speak of for a pi's rasbian. It's progress but not for success.

Some one could design something minimal for what Shoes should do and then see if it can be implemented. NSSound for example, looks minimal. Can that be implemented on Portaudio/Windows and for Linux - pulseaudio appears to be the new standard. That's low on my priorities.

[IanTrudel]

Thanks for the clarification. It's still a valuable progress and we both agree on the priority.

NSSound could be viable. Programming language like Squeak Smalltalk implements sound using native APIs such as NSSound and Win32 Sound API. As a side note, there is a page on building PortAudio on MacOS X.

[ccoupe]

Gstreamer has great documentation and is a level higher than portaudio . Might be too big.

[IanTrudel]

This is it! This is what we need!

GStreamer is compatible with every platform Shoes runs on. Ruby bindings are readily available and current (a rarity in Ruby world).

Basically, we can include GStreamer libraries where it is needed (e.g. Windows) and include Ruby/GStreamer gem in Shoes. Test the whole thing on every platform and then ready to release.

We would get everything we had before (feature wise) and then some. Who would complain!

[ccoupe]

Good find! It's not 'simple' but I'm OK with that. It could also replace VLC for video. Managing the eventloop(s) would be the first challenge, particularly on OSX. Looking at the commits to the rakefle, I believe there are other lessons we could learn.

[ccoupe]

$ gem install gstreamer --no-doc
Fetching: pkg-config-1.3.1.gem (100%)
Successfully installed pkg-config-1.3.1
Fetching: native-package-installer-1.0.6.gem (100%)
Successfully installed native-package-installer-1.0.6
Fetching: glib2-3.2.5.gem (100%)
Building native extensions.  This could take a while...
Successfully installed glib2-3.2.5
Fetching: gobject-introspection-3.2.5.gem (100%)
Building native extensions.  This could take a while...
[sudo] password for ccoupe to install <libgirepository1.0-dev>: 
Successfully installed gobject-introspection-3.2.5
Fetching: gstreamer-3.2.5.gem (100%)
Building native extensions.  This could take a while...
Successfully installed gstreamer-3.2.5
5 gems installed

I wonder what native-package-installer does?

[IanTrudel]

I wonder what native-package-installer does?

Seems to be a helper to install native gems.

https://github.com/ruby-gnome2/native-package-installer