search

marcel-licence / esp32_midi_sampler

GNU General Public License v3.0
74 stars 13 forks source link

ESP8388 - No Sound. How to play samples from buttons or custom Sensors ? #68

Closed baesek closed 2 years ago

baesek commented 2 years ago

EDIT: I think the reason of getting no sound has to to with code from ES8388_Setup() as I get the following output in the serial console on boot:

Connect to ES8388 codec... 0x00: 0x05
[  1071][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
0x01: 0x40
[  1084][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
[  1087][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
[  1099][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
AdcCh1!
MixChAMPL!
[  1117][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
[  1126][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
ES8388 setup finished!
Reg 0x00 = 0x05
[  1149][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...

...
the last line then  repeats 50 times or so

Hi, I have a ESP32 Audio Kit V2.2 2957 board with ES8388 Codec and i just want to make it play a sound through the headphone output (later also through speakers) when i push one of the 6 buttons. To do so I implemented basic button reads by adding `//Edit to config.h:

define AUDIO_KIT_BUTTON_DIGITAL instead of #define AUDIO_KIT_BUTTON_ANALOG`

Then i implemented it as follows in the esp32_audio_kit_module.h file. I receive the button events up and down, but I cannot really play the samples i uploaded the data folders contents to the psram using the mylittlefs plugin for the arduino ide already. I tried to get it act like this: https://www.youtube.com/watch?v=r0af0DB1R68

In the end i want to read sensors and playback samples as they reach a treshold... Edits to z_config.h to implement digital button input : Change 

#ifdef AUDIO_KIT_BUTTON_ANALOG 
to
#if (defined AUDIO_KIT_BUTTON_ANALOG) || (defined AUDIO_KIT_BUTTON_DIGITAL)

Edits to esp32_audio_kit_module.h to implement digital button input :

...
#ifdef AUDIO_KIT_BUTTON_DIGITAL
/*
 * when not modified and R66-R70 are placed on the board
 */

#define PIN_KEY_1                   (36)
#define PIN_KEY_2                   (13)
#define PIN_KEY_3                   (19)
#define PIN_KEY_4                   (23)
#define PIN_KEY_5                   (18)
#define PIN_KEY_6                   (5)
/* My edit */
uint8_t buttonMap[6] = { PIN_KEY_1, PIN_KEY_2, PIN_KEY_3, PIN_KEY_4, PIN_KEY_5, PIN_KEY_6 };
uint8_t buttonStates[sizeof(buttonMap)];
/* End of edit */ 

...

typedef void(*audioKitButtonCb)(uint8_t, uint8_t);
extern audioKitButtonCb audioKitButtonCallback;

...

/*
 * pullup required to enable reading the buttons (buttons will connect them to ground if pressed)
 */
void button_setup()
{
#ifdef AUDIO_KIT_BUTTON_DIGITAL
    // Configure keys on ESP32 Audio Kit board
/* My edit */
    for (uint8_t i = 0; i < sizeof(buttonMap); i++) {
      pinMode(buttonMap[i], INPUT_PULLUP);
    }
/* End of My Edit */
#endif
...
}
...

/*
 * very bad implementation checking the button state
 * there is some work required for a better functionality
 */
void button_loop()
{
#ifdef AUDIO_KIT_BUTTON_DIGITAL
    /* My edit */
    static uint8_t buttonCount = 6;//sizeof(buttonMap);
    static bool state;

    for (uint8_t i = 0; i < buttonCount; i++) {
      state = digitalRead(buttonMap[i]);
      if (state != buttonStates[i]) {
        if (state == LOW) {
          Serial.println("Button "); Serial.print(i+1 ); Serial.println(" down");
          if (audioKitButtonCallback != NULL)
          {
              audioKitButtonCallback(i, 1);
          }
        } else {
          Serial.println("Button "); Serial.print(i+1); Serial.println(" up");
          if (audioKitButtonCallback != NULL)
          {
              audioKitButtonCallback(i, 0);
          }
        }
        buttonStates[i] = state;
      }
    }
    /* End of my edit */
#endif
...
}

#endif
marcel-licence commented 2 years ago

Hi, does the firmware work without any modifications except setting up I2S and I2C to ensure that the codec is working? It might be possible that there are some GPIO conflicts. At the moment I do not have any idea where the error from the Wire library came from. Best regards, Marcel

baesek commented 2 years ago

Hi Marcel, Thank you very much for your answer(time), I just cloned the repo again and I just uncommented

define NOTE_ON_AFTER_SETUP

After upload I get a beep sound on the left ear of my phones. It does not sound like a sample playback more like a rectangle or sawtooth wave. This is what i get on the Arduino Monitor. Note: I set verbose output at compilation and upload in the arduino settings and for the Core Debug Level setting in the board settings.

⸮⸮�⸮�⸮⸮⸮�⸮���⸮�⸮⸮⸮⸮⸮��A⸮zA⸮⸮⸮⸮..⸮⸮⸮⸮0,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0030,len:1344
load:0x40078000,len:13864
load:0x40080400,len:3608
entry 0x400805f0
[�⸮⸮mum⸮⸮⸮⸮2-hal-cpu.c:214] setCpuFrequencyMhz(): PLL: 480 / 2 = 240 Mhz, APB: 80000000 Hz
[   451][I][esp32-hal-psram.c:96] psramInit(): PSRAM enabled

Connect to ES8388 codec... 0x00: 0x05
[  1071][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
0x01: 0x40
[  1084][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
[  1087][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
[  1099][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
AdcCh1!
MixChAMPL!
[  1117][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
[  1126][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
ES8388 setup finished!
Reg 0x00 = 0x05
[  1149][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x01 = 0x40
[  1165][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x02 = 0x00
[  1179][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x03 = 0x00
[  1192][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x04 = 0x3c
[  1206][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x05 = 0x00
[  1220][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x06 = 0x00
[  1234][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x07 = 0x7c
[  1248][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x08 = 0x00
[  1262][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x09 = 0x88
[  1276][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x0a = 0x50
[  1290][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x0b = 0x02
[  1304][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x0c = 0x0c
[  1317][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x0d = 0x02
[  1331][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x0e = 0x30
[  1345][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x0f = 0x20
[  1359][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x10 = 0x00
[  1373][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x11 = 0x00
[  1387][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x12 = 0x16
[  1401][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x13 = 0xb0
[  1415][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x14 = 0x32
[  1429][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x15 = 0x06
[  1442][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x16 = 0x00
[  1456][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x17 = 0x18
[  1470][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x18 = 0x02
[  1484][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x19 = 0x22
[  1498][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x1a = 0x00
[  1512][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x1b = 0x02
[  1526][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x1c = 0x08
[  1540][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x1d = 0x00
[  1554][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x1e = 0x1f
[  1567][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x1f = 0xf7
[  1581][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x20 = 0xfd
[  1595][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x21 = 0xff
[  1609][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x22 = 0x1f
[  1623][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x23 = 0xf7
[  1637][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x24 = 0xfd
[  1651][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x25 = 0xff
[  1665][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x26 = 0x1b
[  1679][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x27 = 0xb8
[  1692][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x28 = 0x28
[  1706][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x29 = 0x28
[  1720][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x2a = 0xb8
[  1734][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x2b = 0x80
[  1748][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x2c = 0x00
[  1762][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x2d = 0x00
[  1776][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x2e = 0x1e
[  1790][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x2f = 0x1e
[  1804][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x30 = 0x1e
[  1817][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x31 = 0x1e
[  1831][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x32 = 0x00
[  1845][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x33 = 0xaa
[  1859][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Reg 0x34 = 0xaa
[  1873][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
[  1877][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
[  1889][E][Wire.cpp:313] beginTransmission(): Unfinished Repeated Start transaction! Expected requestFrom, not beginTransmission! Clearing...
Setup Serial2 with 31250 baud with rx: 21 only
Setup MidiPort2 using Serial2
Total PSRAM: 4192107
Free PSRAM: 4191587
Try to allocate 4191586 bytes
Not able to allocate the complete PSRAM buffer!
Now trying to reduce the allocation buffer sizeTotal PSRAM: 4192091
Free PSRAM: 62815
Allocated 4128756 bytes
storageLen: 2064378
storageLen: 46.81s
rev: 15329, 15329
Reverb is ready!
delay samples: 11025
Max delay time: 0.25s
ESP.getFreeHeap() 143464
ESP.getMinFreeHeap() 143416
ESP.getHeapSize() 176692
ESP.getMaxAllocHeap() 110580
Total heap: 176692
Free heap: 143464
Total PSRAM: 4192059
Free PSRAM: 18679
Firmware started successfully
[?25lLoaded sample: 22049            
--------------------------------
inL              outL   ####    
inR              outR   ####    
--------------------------------
56926 of 2064378 bytes used     
O_______________________________
                                LittleFS: pet_bottle.wav                                        
--------------------------------
Key1     Key3     Key5     #####
#### Key2     Key4     Key6

What do you mean be setting up I2S and I2C? I couldn't find any relating setting in the config.h nor in the README.md

I dont know if this is relevant, but: The Key2 on my board is only working if set Dip Switches 1, 2, 3 to ON and 4,5 to OFF, If I set only 2 and 3 to ON it is not working. (detectable by my test code)

Thank you for any help on this. I am so curious to hear some sound coming from the board. all the best! -paul

EDIT: I just found out that the Board won't output audio if I set KEY6 (Pin 5) as INPUT_PULLUP So if i take out that pin from my custom implementation of the tact switches I also get the beep and can play quite high notes with the buttons after clicking around on KEY1 for a while. Is there a documentation of the Serial menu ? I didnt get how it works so far.

//uint8_t buttonMap[6] = { PIN_KEY_1, PIN_KEY_2, PIN_KEY_3, PIN_KEY_4, PIN_KEY_5, PIN_KEY_6 };
uint8_t buttonMap[5] = { PIN_KEY_1, PIN_KEY_2, PIN_KEY_3, PIN_KEY_4, PIN_KEY_5 };
uint8_t buttonStates[sizeof(buttonMap)];
...
/*
 * pullup required to enable reading the buttons (buttons will connect them to ground if pressed)
 */
void button_setup()
{
#ifdef AUDIO_KIT_BUTTON_DIGITAL
    // Configure keys on ESP32 Audio Kit board
    for (uint8_t i = 0; i < sizeof(buttonMap); i++) {
      pinMode(buttonMap[i], INPUT_PULLUP);
      buttonStates[i] = HIGH;
    }
#endif
#ifdef AUDIO_KIT_BUTTON_ANALOG_OLD
    adcAttachPin(PIN_KEY_ANALOG);
    analogReadResolution(10);
    analogSetAttenuation(ADC_11db);
#endif
}

Can you explain how to play back a sound file form LittleFS as one shot sample from code in the main loop? Like:`play('sound.wav'); delay('1000'); and further how to setup the sample playback settings from code (tune, reverb, adsr, looping etc.)? Edit: Sorry I just realised that there is a Discussions area which might answer this question. I ll try to find it out there

marcel-licence commented 2 years ago

Hi, the loop area of the Demo sound might be sound a bit odd. I am not sure if there is a problem or just the odd sound.

To playback sounds from LittleFS you can use the drumcomputer project which is capable of a direct playback. Using the sampler project you need to load the sample into the PSRAM using the code like this:

    PatchManager_SetDestination(0, 1);
    Sampler_LoadPatchFile("/samples/pet_bottle.wav");

The settings for the sample will fall back to default if no binary file can be found containing the information of a loop point etc. The sound will be assigned to the next free channel. In addition to that channel 0 has one sound assigned for each key.

In https://github.com/marcel-licence/esp32_midi_sampler/blob/main/z_config.ino you will find a lot of connected functions which are used to control the sampler. There was a discussion in the past with some details how to set up a sound without any midi controller.

Best regards, Marcel

baesek commented 2 years ago

Hi Marcel, thank you ! now I understood the difference between the drumcomputer and the sampler project. Do you know the difference in latency the midi_sampler has in relation to the drumcomputer as it plays back the samples from PSRAM instead of the LittleFS? Again, just for understanding the two concepts better: The drumcomputer can can have a maximum of 4MB (around 45s of 16bit 44100 mono audio) split into maximum of 8 samples readily available for low latency playback directly from the littleFS ?

The midi sampler sampler loads a maximum of also 4MB from either sd card or the littleFs to the PSRAM for playback. The advantage is, that it can have a collection of samples on the sdcard which then can be loaded into the PSRAM when needed. I suppose this is not happening fast anough to trigger sounds from sd card directly without noticable latency.

I am just rethinking if the littleFs can really hold 4MB or is it 4MB - size of program? I am seeing the Partition scheme Setting of the Arduino Ide: Default 4MB with Spiffs (1.2MB App / 1.5MB SPiffs) Does this mean maximum flash memory for samples is also 1.5MB (15.8 seconds) ?

marcel-licence commented 2 years ago

SPIFFS does contain a kind of file system. So you would have a bit less memory of raw data. In case of storing 16bit 44100 samples it would end up by approx 15 seconds. Regarding the SD card, yes streaming is very limited. In addition to that you would get problem when reading multiple file at once. I worked on this topic when doing the mellotron like sampler project: https://youtu.be/U5Q8chfMglE From SPIFFS you can read multiple files at once and finally PSRAM behave more like RAM. Its much faster.