Play two wav files simultaneously?

[dagnall53]

Do you think it would be possible to play two wav files simultaneously? and asynchronously. (at least as far as when they individually start and finish...The sample rates would be the same) I tried some of the obvious things like having two definitions of wav and file, but there is only one "out" ---

[earlephilhower]

This requires a mixer output element. Conceptually simple, but just not something I've added just yet. Let's keep this open so I can be reminded when some random free time appears...

[dagnall53]

That would be great.. I will wait patiently! Have a Peaceful Christmas!.

Dagnall (I've had complaints from the better half about persistent "chuffs" so will be laying low for a while!...)

[dagnall53]

FYI I have re-organised my Test code and popped it up onto Github. https://github.com/dagnall53/LocoSoundEffects It still (very) occasionally starts in a noisy state.. But mainly it works fine. I think my "file hygiene" is OK, and I am closing files when they have played.

If you try it, the whistle seems to overload the driver in places, but I can reduce the volume later. Video https://youtu.be/Vd0HbV_MXVI

Looking forward to being able to upgrade with the mixer function...

Dagnall

[earlephilhower]

Neat. Looks like you had some time over the holiday to tinker.

Just add a out->SetGain(0.5); after the out object creation and it will lower the volume for you to 50%.

[ErichHeinemann]

When using the DAC98357A with 3.3V as VSS, some people mentioned to use an extra resistor in line with the DIN-PIN. I found this info in a german project. The Voltage for DIN should be lower than VSS. With this DAC, out->SetGain(0.4); is the limit before getting overload.

[earlephilhower]

This is unexpected. Are they running them both from the same 3.3V source? A resistor going from an CMOS output to a CMOS input will slow down the signal transition a bit (i.e. it becomes an RC exponential attack function), but shouldn't actually change the final voltage at all since in steady state there is no current flowing. CMOS->CMOS just has to charge/discharge the base of a couple transistors...after that it's basically only leakage current you need to keep supplied.

If they're using different VCCs, and, say, the ESP8266 is 3.6 (10% high) and the I2S is 3.0 (10% low) then I can see this helping avoid any sort of latch-up through the input protection diodes since CMOS pads really don't like to have voltages higher than their own VCC present.

[dagnall53]

Interesting, I have the DAC running on 5v, and have the Din connected direct. I have had some occasional startups where the sound has been over loud and distorted. I was trying to home in on this, suspecting a startup issue. I have just added a 32K resistor in series with Din (I just had it on the desk, nothing selected about it) and the sound still works fine, So far, it has also powered up fine with no repeat of the "overload" I had occasionally seen before. -- we will see...

[ErichHeinemann]

Hi, 5Volts for VSS and 3.3 for DIN should be ok. only 3.3 for VSS and 3.3 for DIN could be a problem.

I have seen this in this discussion:

https://www.heise.de/ct/ausgabe/2017-22-Eine-portable-Soundmachine-auf-Basis-des-Espressif-ESP32-3851746.html https://www.heise.de/ct/ausgabe/2017-22-Eine-portable-Soundmachine-auf-Basis-des-Espressif-ESP32-3851746.html

... dadwmm // 31.12.2017 01:36 Krächzen … und Unterbrechung des Stückes Hallo,

Krächzen: Wenn wie gezeigt aufgebaut ist der Ton übel. Knackser, Krächzen - zum einstampfen.

Bitte im "Schaltplan" berücksichtigen, dass der MAX an der Spannungsversorgung ein höheres Potential braucht als am digitalen input. Siehe auch Datenblatt. Am einfachsten die Spannungsversorgung über 5V Pin des ESP32, oder wie im Datenblatt vorgesehen einen Widerstand vor den digitalen input wenn über 3.3V gespeist.

…

translated .. . referring the documentation, the DAC needs a higher VSS than the DIN-Input. .. thats all.

Regards Erich

Am 30.01.2018 um 12:26 schrieb dagnall notifications@github.com:

Interesting, I have the DAC running on 5v, and have the Din connected direct. I have had some occasional startups where the sound has been over loud and distorted. I was trying to home in on this, suspecting a startup issue. I have just added a 32K resistor in series with Din (I just had it on the desk, nothing selected about it) and the sound still works fine, So far, it has also powered up fine with no repeat of the "overload" I had occasionally seen before. -- we will see...

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/earlephilhower/ESP8266Audio/issues/24#issuecomment-361564872, or mute the thread https://github.com/notifications/unsubscribe-auth/AiOjNUPJR0uYAGDuyECQfy7Rw--HPSy9ks5tPvx2gaJpZM4RLPjq.

[earlephilhower]

https://datasheets.maximintegrated.com/en/ds/MAX98357A-MAX98357B.pdf

After reading the datasheet I'm still not seeing anything about needing interface inputs below VDD. Page 17 talks about keeping SD_MODE < VDD, but that's a simple jumper here from VDD directly.

What this is really is probably a setup/hold time violation coming from the ESP8266. That would explain the over-amplification: because a setup or hold time is too small, the DAC misses the 1st bit of audio data. That then effectively cause it to shift-left the audio data by 1, so when you tried to send +5000 it actually reads and amplifies as +10000. Adding a resistor somewhere would slow down a transition just enough to meet the specs and you're golden. In any case, it's a marginal design somewhere that a oscilloscope would be needed to diagnose exactly since we're talking about 10ns scale for setup requirements.

I can add a note about this in the README. Seems like only when you're running the MAX98357A off the same 3.3V supply as the ESP8266, and you only need to slow down the DIN pin, right?

[dagnall53]

I was running the dac from 5v, and as I said, the fault was very intermittent, but your suggestion about the slight delay sounds very reasonable. My random choice of a fairly large resistor may have been helpful.

The comment about gain 0.4 also seems appopriate. I used audacity to make all my sound clips max out at-3db but was still getting some overload. I've now got individual volume controls set for all my sound effects so that they all sound ok.

[dagnall53]

Earle, whilst browsing, I came across this library: https://github.com/TMRh20/TMRpcm it seems to have a "Multiplay" option, with two "drives" to one output (?). Is this concept of any help in getting a "mixer", or to get two wavs to play together?

[ErichHeinemann]

Hi, I have added a Mixer based on 2 classes, a AudioMixerInBuffer which is more or less one Mixer-Channel and the final AudioMixerOutBuffer which is the main-channel or "Master-Channel" of a Mixer. https://github.com/ErichHeinemann/hman-stomper/tree/master/AdditionalLibs/ESP8266Audio .. copy the 4 needed files with the Names "AudioMixer.." over into Your Libraries... A functional example is stored at: https://github.com/ErichHeinemann/hman-stomper/tree/master/TestProjects/Mixer/PlayWAVFromPROGMEMWithMixerDAC I have used my custom AudioGeneratorSAMPLE which adds Velocity and Pitch to the generator which works for percussion sounds acceptable. You could add some Sounds using any valid AudioGeneratorXXX.

[dagnall53]

Erich, Many thanks for you work, I have sent a Email detailing some issues I have with SPIFFS and WAV .. Hopefully it will not get lost in your junk/spam folder, and you can tell me where I am going wrong.. Dagnall

[dagnall53]

@ErichHeinemann I have put a (non functioning) simple SPIFFS + WAV +NODAC example in https://github.com/dagnall53/Mixertest/blob/master/mixertest.ino (https://github.com/dagnall53/Mixertest/blob/master/mixertest.ino) Hopefully you may be able to get it working?

[ErichHeinemann]

I got Your email and already responded to it.

Best Regards Erich

Am 19.02.2018 um 19:24 schrieb dagnall notifications@github.com:

Erich, Many thanks for you work, I have sent a Email detailing some issues I have with SPIFFS and WAV .. Hopefully it will not get lost in your junk/spam folder, and you can tell me where I am going wrong.. Dagnall

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/earlephilhower/ESP8266Audio/issues/24#issuecomment-366771180, or mute the thread https://github.com/notifications/unsubscribe-auth/AiOjNexE8TtRXPXWP-HLoTPhHeB9pjxZks5tWbxagaJpZM4RLPjq.

[dagnall53]

Thanks, I will explore more tomorrow D

Sent from my iPhone

On 19 Feb 2018, at 20:10, Erich notifications@github.com wrote:

I got Your email and already responded to it.

Best Regards Erich

Am 19.02.2018 um 19:24 schrieb dagnall notifications@github.com:

Erich, Many thanks for you work, I have sent a Email detailing some issues I have with SPIFFS and WAV .. Hopefully it will not get lost in your junk/spam folder, and you can tell me where I am going wrong.. Dagnall

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/earlephilhower/ESP8266Audio/issues/24#issuecomment-366771180, or mute the thread https://github.com/notifications/unsubscribe-auth/AiOjNexE8TtRXPXWP-HLoTPhHeB9pjxZks5tWbxagaJpZM4RLPjq.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[dagnall53]

I have moved comments on Erich's mixer across into "Issues" in Erich's code : https://github.com/ErichHeinemann/hman-stomper/issues/1 Hopefully when its working fully , Earle can add it to his "working" library.

[earlephilhower]

I looked at Erich's code, but couldn't quite understand some of the choices he made. They're probably fine for his use case, but I think a more general solution is needed. I've pushed a new class, AudioOutputMixer, which works, more-or-less. The sample code plays a WAV twice, with one playback starting 2 seconds after the first.

Basically you need to make the mixer, then for each Generator do stub = mixer->NewInput(), pass that stub to the Generator, and make sure to call stub->stop when you call Generator->stop. Be sure to call all Generator->loop() in your main loop, of course.

It keeps a single buffer, at 32bpp, which is then xverted to 16 bits, as normal, to avoid wraparound problems. The SetVolume() calls are not implemented, but when there will need to be on each stub input (i.e. one input can be vol=0.4, another at vol=1.5).

Like the commit says, it's WIP, so expect changes as it finalizes. The main flow should be the same...

[earlephilhower]

Volume on each stub is working now. See the sample (just stub->SetGain()).

[dagnall53]

Many thanks. I will explore this next week and see how it works with SPIFFS.

[earlephilhower]

For now, please make sure the sample rates, channels, and bits-per-sample of all mixed inputs are the same. That bit's not in yet. All but sample rate are simple mechanical changes, really, just have to promote everything to 16b, stereo in the stubs. Sample rate conversion is thornier. You can use the FilterAndDecimate generator which can do this and even perform proper low-pass filtering, but a simple decimation should be added at some point to the muxer itself.

[dagnall53]

[edited, as I have found the bug this referred to!..]

[dagnall53]

Looks like I can find bugs.. when they are my own.. I did not appreciate that the stubs could be set once in setup. It was my repeated calls to "stub[Channel] = mixer->NewInput();" in my "beginplay" code (now removed to setup) that was causing the issue.

WORKING code, Just FYI and in case you spot anything else I have done wrong..

void Beginplay(int Channel,const char *wavfilename, float Volume){
 // String Filename;
 // Filename=wavfilename;
  Serial.printf("CH:");
  Serial.print(Channel);
  Serial.printf("  Playing <");
  Serial.print(wavfilename);  
  stub[Channel]->SetGain(Volume);
  wav[Channel] = new AudioGeneratorWAV();
  delete file[Channel]; // housekeeping ?
  file[Channel]= new AudioFileSourceSPIFFS(wavfilename);
  //file[Channel] = new AudioFileSourcePROGMEM( viola, sizeof(viola) );  retained for test purposes 
  wav[Channel]->begin(file[Channel], stub[Channel]);
  Running[Channel]=true;
  Serial.printf("> at volume :");
  Serial.println(Volume);
  start[Channel]=millis();
}

void setup()
{
  WiFi.mode(WIFI_OFF); 
  Serial.begin(115200);
  out = new AudioOutputI2SNoDAC();
  mixer = new AudioOutputMixer(32, out);
  stub[0] = mixer->NewInput();
  stub[1] = mixer->NewInput();
  delay(100);
  vol=0.3; 
  Beginplay(1,"/piz11.wav", 1.0);
  Beginplay(0,"/viochrom11.wav", 0.3);
  Play=1;
}

void loop()
{

  if (wav[0]->isRunning()) {
    if (!wav[0]->loop()) { wav[0]->stop(); stub[0]->stop();
    Running[0]=false;Serial.printf("stopping 0\n"); }
  }

  if (wav[1]->isRunning()) {
      if (!wav[1]->loop()) { wav[1]->stop(); stub[1]->stop(); Running[1]=false;Serial.printf("stopping 1\n");}
    }

  if (((millis()-start[1]) > 2000)&& (!wav[1]->isRunning())) {
     if (!Running[1]) {
      Play=Play+1;
      Serial.println(Play);
      Beginplay(1,"/viochrom11.wav", 0.5);
      }
  }

  if (((millis()-start[0]) > 2000)&& (!wav[0]->isRunning())) {
     if (!Running[0]) {
      Play=Play+1;
      Serial.println(Play);
      //Beginplay(0,"/piz11.wav", vol);
      Beginplay(0,"/1000Hz_-3dBFS_3s11k.wav", vol);
      vol=vol*1.2;
      }
  }
}

[earlephilhower]

@dagnall53 To simplify things, I set up up to 8 mixer slots. When you're done with one you should "delete stub[x]" to free that slot, or you can re-use the stub, which should "just work" but is not tested by me. When it's out of slots, the mixer returns NULL which caused your Exception.

So you can do what you're doing in the updated code, or use the original one and right after stop() do a delete stub[..] call which'll free the mixer slot so the next NewInput() call will succeed.

[dagnall53]

Thanks.. I think I will stick to two stubs, set up in Setup(). But I like what you have done and for me I do not need more than 2 inputs and thought I did not need to turn them off.

I have done some tests with 11K and 44k (both mono) samples and I think the ESP is getting a bit loaded up with just two 44k stubs. With 11k samples and two stubs I was able to add up to 5ms delay in the main loop before horrid things happened. Larger buffer size helped. IMHO The audio code should leave at least a few ms, and preferably 10ms in every loop for other things to happen...maybe a larger buffer size might be useful for generic use?

RE HOUSEKEEPING: I'm interested in / have been testing/ the "delete file" housekeeping thing. I put a "delete file[Channel]" in front of the "file[Channel]= new AudioFileSourceSPIFFS(wavfilename);", which I expected to fail first run, but it compiles and runs .... BUT:
Using I2SDAC it got to 510 plays before Exception 29, so there is still something in there... When I removed the "delete file" for tests the program overflows at 309 plays, so the delete file before new seems to do something positive, even if its not the full answer. ...

With I2SNoDAC.. with delete, I got to 510 plays (again) before Exception (29), same as DAC result. (Not trying without delete, I expect an earlier failure..)

Reworking my "stubs" so I delete stub[channel] and have a new for every play... (code below) I get to 475 plays before..Exception 29.. so there is something in here that needs finding/cleaning. I may be missing something, or is there perhaps something in the new mixer code overflowing? perhaps something in the new mixer code overflowing?

Beginplay code segment:

delete stub[Channel];
  stub[Channel] = mixer->NewInput();
  stub[Channel]->SetGain(Volume);

  delete file[Channel]; // housekeeping ?
  file[Channel]= new AudioFileSourceSPIFFS(wavfilename);

  wav[Channel] = new AudioGeneratorWAV();
  wav[Channel]->begin(file[Channel], stub[Channel]);

A subsidary question: since the "delete file" seems to already have a test for something being open (or I would expect it to crash first run) , why not include a "delete file" automatically as part of the "new", as a matter of course, so that every "new" starts clean?

Also, and very minor, you have missed AudioOutputI2SDAC KEYWORD1 from keywords.txt.. (I had to find something sort of helpful!!)

[dagnall53]

I think the issue was with the wav: I have added delete wav before the new wav. and the revised Begin play has been playing now for over 916 loops. (super boring, but welcome!)

Q: given this housekeeping thing would it be possible to add an "automatic" "delete" as part of every "new" ? or are there conditions where this would not be helpful?

Not expecting any quick answers.. Dagnall

``void Beginplay(int Channel,const char *wavfilename, float Volume){
  Serial.print(Play);
  Serial.printf(" CH:");
  Serial.print(Channel);
  Serial.printf("  Playing <");
  Serial.print(wavfilename);  
  delete stub[Channel];
  stub[Channel] = mixer->NewInput();
  stub[Channel]->SetGain(Volume);

  delete file[Channel]; // housekeeping ?
  file[Channel]= new AudioFileSourceSPIFFS(wavfilename);
  delete wav[Channel];
  wav[Channel] = new AudioGeneratorWAV();
  wav[Channel]->begin(file[Channel], stub[Channel]);
  Running[Channel]=true;
  Serial.printf("> at volume :");
  Serial.println(Volume);
  start[Channel]=millis();
  Play=Play+1;  // test for long numbers of repeats for housekeeping checks: delete file issues etc. 
}
``

[earlephilhower]

Keeping existing mixer stubs is great, avoids memory fragmentation.

Auto-delete is not do simple or possible, in general. Normal C++ semantics are that when you new something, you need to delete it later on or else you leak memory, as shown above. It's just the nature of C++, you get to do your own garbage collection.

There is a very cool std::template helper known as std::unique_ptr, which may help you but it's a bit advanced. Instead of new'ing, you use a helper macro which wraps the new'd pointer in a helper class. Then, whenever you assign NULL or overwrite that pointer, it will automatically delete[] the memory associated, freeing it w/o you needing to. It's most useful for when you allocate and need to deallocate on variable scope exit, but it may make sense here. But if you're not friendly w/C++11 it's scary.

For playback underflow, the issue is even if you had a 15 second buffer, if you didn't call the ->loop() before the small I2S DMA chain underflowed, it would still hiccup. The mixer buffer != the DMA buffer.

What seems to be needed here is a mixer->loop() call which simply tries to push I2S samples out. That function could be called many times in a single Arduino loop{} or other long-running function, and as long as there was < xxms (depends on sample rate) between calls you'd be OK.

Alternatively, some sort of IRQ callback might make sense. I don't know how well those are supported on the ESP8266, and there are serious limitations on what you can do in them, but if you got an IRQ on buffer empty, it would allow you to quickly stuff data into the free'd buffer...

[earlephilhower]

Oh, and also SPIFFs can be very slow in some circumstances. So if your samples can fit in PROGMEM, try that instead and you'll see some serious speed improvements.

[dagnall53]

Many thanks for the advice, I will try and see if I can simplify the delete stuff tomorrow. And perhaps explore exactly how much "delay" can be used. (Not that I add delay in the target code, but there is a lot actual wifi "work" going on.. ) On SPIFFS, I chose them because of the 1M code limitation in Arduino. I would not mind having stuff in progmen if i could use the whole 4Mb.. But I understand this is an arduino / memory paging issue. My "target" code has a built in FTP server, so changing the main SPIFF files remotely is a doddle.. If I can retain it I will.. But I will also look and see if I can fit my key (often called chuff sounds ) samples in progmem and still retain the ability to update the code over wifi using OTA updating. I may end with a hybrid of progmem for frequent and short sounds, and SPIFFS for longer sounds.

Mixer->loop. I had wondered if this might be a good idea. your (and Earles) codes that do everything via the wav->loop are simple in concept and application, but I had wondered if having a mixer->loop might ease some things by allowing the mixer buffers to be filled as part of the wav->loop and then emptied by the mixer->loop . Unfortunately that is too scary for me to try and code.. And your code is working .. which is a very big reason for not changing! All the best Dagnall

[earlephilhower]

There's no need for mixer->loop(), I completely blanked there. Just call either or both wav1->loop(), wav2->loop() in your long-running functions. Either one will push all samples it can to I2S, and if there's buffer space try and fill it. If there's no space in the buffer or in I2S, these return immediately and shouldn't take any appreciable time at all.

[earlephilhower]

Just FYI, I did add that mixer->loop() method. While I'd still recommend calling all the wav[]->loops() instead, you can call the mixer->loop() to only send out pre-computed samples (i.e. it's faster/shorter than the wav loop() calls).

[dagnall53]

Many thanks for your help on this.. Using SPIFFS turns out to be fast enough in my application, and now I can "toot" whilst I simultaneously "chuff" I have uploaded a short video to show it working,.
https://www.youtube.com/watch?v=Pg5r6MZDhww

Many thanks again. Dagnall

[earlephilhower]

That's pretty cool. You should write it up for other folks, it seems like something lots of folks would like but probably costs a fortune to get commercially.

[dagnall53]

Thanks for the encouragement. For anyone else interested:
My Loco and Wagons are on Thingyverse where they have their own write ups: https://www.thingiverse.com/thing:2744624 and https://www.thingiverse.com/thing:2689057 The ESP8266 code is on GitHub: https://github.com/dagnall53/ESPMQTTRocnetSound There is a long write up about the code on the Github. To make it all work, you need RocRail, http://wiki.rocrail.net/doku.php which is not entirely free, but needs a small subscription about €12 a year to get all the bits like the Tablet web stuff working. (I have no affiliation to Rocrail.. I'm just a user.. ) All the best and thanks again Dagnall

[earlephilhower]

I've added a link to yours and @ErichHeinemann 's projects in the readme.md file. Closing this since I think we're basically done. The rest is optimization. :)

[ghost]

If someone looking for @dagnall53 complete code with include etc: @dagnall53 s Sound files from his https://github.com/dagnall53/ESPMQTTRocnetSound project are just awesome :)

#include "AudioGeneratorWAV.h"
#include "AudioOutputI2S.h"
#include "AudioFileSourceSPIFFS.h"
#include "AudioOutputMixer.h"
#include <WiFi.h>
AudioOutputMixerStub *stub[2];
AudioOutputMixer *mixer;
AudioGeneratorWAV *wav[2];
AudioFileSourceSPIFFS *file[2];
AudioOutputI2S *out;
boolean Running[2];
long start[2];
long Play = 0;
void Beginplay(int Channel, const char *wavfilename, float Volume) {
  // String Filename;
  // Filename=wavfilename;
  Serial.printf("CH:");
  Serial.print(Channel);
  Serial.printf("  Playing <");
  Serial.print(wavfilename);
  stub[Channel]->SetGain(Volume);
  wav[Channel] = new AudioGeneratorWAV();
  delete file[Channel]; // housekeeping ?
  file[Channel] = new AudioFileSourceSPIFFS(wavfilename);
  //file[Channel] = new AudioFileSourcePROGMEM( viola, sizeof(viola) );  retained for test purposes
  wav[Channel]->begin(file[Channel], stub[Channel]);
  Running[Channel] = true;
  Serial.printf("> at volume :");
  Serial.println(Volume);
  start[Channel] = millis();
}

void setup()
{
  WiFi.mode(WIFI_OFF);
  Serial.begin(115200);
  out = new AudioOutputI2S();
  mixer = new AudioOutputMixer(32, out);
  stub[0] = mixer->NewInput();
  stub[1] = mixer->NewInput();
  delay(100);
  Beginplay(1, "/F1.wav", 1.0);
  Beginplay(0, "/F2.wav", 0.3);
  Play = 1;
}

void loop()
{

  if (wav[0]->isRunning()) {
    if (!wav[0]->loop()) {
      wav[0]->stop(); stub[0]->stop();
      Running[0] = false; Serial.printf("stopping 0\n");
    }
  }

  if (wav[1]->isRunning()) {
    if (!wav[1]->loop()) {
      wav[1]->stop();
      stub[1]->stop();
      Running[1] = false;
      Serial.printf("stopping 1\n");
    }
  }

  if (((millis() - start[1]) > 2000) && (!wav[1]->isRunning())) {
    if (!Running[1]) {
      Play = Play + 1;
      Serial.println(Play);
      Beginplay(1, "/F1.wav", 0.5);
    }
  }

  if (((millis() - start[0]) > 2000) && (!wav[0]->isRunning())) {
    if (!Running[0]) {
      Play = Play + 1;
      Serial.println(Play);
      //Beginplay(0,"/piz11.wav", vol);
      Beginplay(0, "/F2.wav", 0.5);
    }
  }
}

[earlephilhower]

I've added a link on the top of the README to his and Erich's projects, so hopefully it's easier to find.

@dagnall53 may want to put a note on the ESP8266 forum if he wants to get some traffic.

[dagnall53]

Many thanks. I have added a new topic on the ESP8266 forum. under General discussions... http://www.esp8266.com/viewtopic.php?f=6&t=17207&p=0&e=0&sid=9d624008d0a7fcaa219f5101a97a8d0f . I attached an article about the project I had written for a UK garden railway magazine, but which was not published.