search

jorgenkraghjakobsen / snapclient

WIP snapclient on ESP32
237 stars 48 forks source link

Using the MAX98357A #43

Open Aragur opened 2 years ago

Aragur commented 2 years ago

Hey, I'm pretty stuck right now on implementing this snapclient on my own hardware.
I'm using a MAX98357A based on this example.
Currenlty all configuration of audio devices isn't working. (I tried all devices on the "Custom Audio Board"). After research (I'm a complete noob when it comes to audio streaming) I found out that I have to add my own driver for the Audio HAL module. But I have no idea how to get this, or how to write my own. Is it possible to stream the audio using this library?: https://github.com/schreibfaul1/ESP32-audioI2S Or this library: https://github.com/earlephilhower/ESP8266Audio (as it supports more audio codecs) Or ist there a pretty basic solution to stream PCM audio directly to my I2S board?

Thanks a lot for your help. I'm pretty much locked in as I bought a lot of these boards for a multiroom audio solution, and a only found out about this project as I noticed that an Icecast server wont be enough)

CarlosDerSeher commented 2 years ago

@Aragur You probably should study the files in https://github.com/jorgenkraghjakobsen/snapclient/tree/master/components/custom_board/pcm51xx as a reference. Copy it as a new board and rename everything to reflect your codec. All the i2c stuff can be removed as you don't need it. You'll probably end up with some empty functions and some gpio configuration. Study the datasheet of your codec for details on it's initialization and configure esp's i2s interface in accordance. I am not sure how this specific codec will work out as it hasn't a mclk signal but it should probably work without issues. You realize that you bought a mono IC? You would need two to get a stereo signal, again study the datasheet for details.

I didn't take a closer look, but I think the linked libraries are of no use here.

Regards Karl

majoreffort commented 2 years ago

@Aragur I actually added a TI PCM5102A board several weeks ago and just confirmed today this also works with a MAX98357A, so you should be able to use this out of the box. In fact, this should work with about any pure I2S DAC without I2C communication/setup requirements.

You can even connect SD_MODE the same way as the X(SMT) pin on PCM5102A boards to have power down/mute capability on the board (plus a resistor if you want to mix stereo channels instead of only using the left channel, see datasheet).

See my code in https://github.com/majoreffort/snapclient/blob/NETCONN/components/custom_board/pcm5102a/pcm5102a.c Note: this is based on @CarlosDerSeher's fork NETCONN branch.

Not sure if I should make a pull request to this repo or your fork @CarlosDerSeher ?

Aragur commented 2 years ago

Thanks for all your effort.

@majoreffort By using your NETCONN branch I cannot connect to my server anymore. Anyhting I can do to fix this? SC: can't connect to remote 192.168.178.51:1704, err -13

Aragur commented 2 years ago

So now I got it working by porting the changes to this codebase. My audio signal is pretty bad as my ring buff has an error all the time reading audio data. I'm using a PCM stream as Opus is crashing my board directly on initialization. Here is a part of my log:

I (1762) I2S: I2S interface master setup
I (1772) I2S: Setup i2s dma and interface
Setup ringbuffer using internal ram - only space for 150ms - Snapcast buffer_ms parameter ignored
Ringbuffer ok
I (1782) SNAPCAST: ... allocated socket
I (1812) SNAPCAST: ... connected
I (1832) SNAPCAST: Buffer length:  150
I (1832) SNAPCAST: Ringbuffer size:28800
I (1832) SNAPCAST: Latency:        0
I (1832) SNAPCAST: Mute:           0
I (1832) SNAPCAST: Setting volume: 100
I (1842) SNAPCAST: Received codec header message

I (1842) SNAPCAST: Codec : pcm , Size: 44

I (1872) I2S: 150 49 syncing ...
I (1892) I2S: 150 69 syncing ...
I (1912) I2S: 150 89 syncing ...
I (1932) I2S: 150 109 syncing ...
I (1952) I2S: 150 129 syncing ...
I (1972) I2S: 150 149 syncing ...
I (1992) I2S: 150 169 syncing ...
I (1992) I2S: 150 169 SYNCED
I (1992) I2S: Chunk :3840 152 ms
I (2072) I2S: Error readding audio from ring buf : read 1940 of 3840 , missing 1900
I (2072) I2S: Read the next 1900
I (2242) I2S: Error readding audio from ring buf : read 40 of 3840 , missing 3800
I (2242) I2S: Read the next 3800
I (2402) I2S: Error readding audio from ring buf : read 1992 of 3840 , missing 1848
I (2402) I2S: Read the next 1848
I (2572) I2S: Error readding audio from ring buf : read 92 of 3840 , missing 3748
I (2572) I2S: Read the next 3748
I (2732) I2S: Error readding audio from ring buf : read 2044 of 3840 , missing 1796
I (2732) I2S: Read the next 1796
I (2902) I2S: Error readding audio from ring buf : read 144 of 3840 , missing 3696
I (2902) I2S: Read the next 3696
I (3062) I2S: Error readding audio from ring buf : read 2096 of 3840 , missing 1744
I (3062) I2S: Read the next 1744
I (3232) I2S: Error readding audio from ring buf : read 196 of 3840 , missing 3644
I (3232) I2S: Read the next 3644
I (3392) I2S: Error readding audio from ring buf : read 2148 of 3840 , missing 1692
I (3392) I2S: Read the next 1692
I (3562) I2S: Error readding audio from ring buf : read 248 of 3840 , missing 3592
I (3562) I2S: Read the next 3592
I (3722) I2S: Error readding audio from ring buf : read 2200 of 3840 , missing 1640
I (3722) I2S: Read the next 1640
I (3872) I2S: Chunk :3840 42 ms
I (3892) I2S: Error readding audio from ring buf : read 300 of 3840 , missing 3540
I (3892) I2S: Read the next 3540
I (4042) I2S: Error readding audio from ring buf : read 2252 of 3840 , missing 1588
I (4042) I2S: Read the next 1588
I (4212) I2S: Error readding audio from ring buf : read 352 of 3840 , missing 3488
I (4212) I2S: Read the next 3488
I (4382) I2S: Error readding audio from ring buf : read 2304 of 3840 , missing 1536
I (4382) I2S: Read the next 1536
I (4552) I2S: Error readding audio from ring buf : read 404 of 3840 , missing 3436
I (4552) I2S: Read the next 3436
I (4712) I2S: Error readding audio from ring buf : read 2356 of 3840 , missing 1484
I (4712) I2S: Read the next 1484
I (4892) I2S: Error readding audio from ring buf : read 456 of 3840 , missing 3384
I (4892) I2S: Read the next 3384
I (5052) I2S: Error readding audio from ring buf : read 2408 of 3840 , missing 1432
I (5052) I2S: Read the next 1432

This is the crash when using opus: (Returning error -7 which translates to OPUS_ALLOC_FAIL)

I (1762) I2S: I2S interface master setup
I (1772) I2S: Setup i2s dma and interface
Setup ringbuffer using internal ram - only space for 150ms - Snapcast buffer_ms parameter ignored
Ringbuffer ok
I (1782) SNAPCAST: ... allocated socket
I (1982) SNAPCAST: ... connected
I (2002) SNAPCAST: Buffer length:  1000
I (2002) SNAPCAST: Ringbuffer size:192000
I (2002) SNAPCAST: Latency:        0
I (2002) SNAPCAST: Mute:           0
I (2012) SNAPCAST: Setting volume: 100
I (2012) SNAPCAST: Received codec header message

I (2022) SNAPCAST: Opus sampleformat: 48000:16:2

I (2022) SNAPCAST: Failed to init opus coder
E (2032) FreeRTOS: FreeRTOS Task "http_get_task" should not return, Aborting now!

abort() was called at PC 0x4008d02b on core 1
0x4008d02b: vPortTaskWrapper at /Users/giesel/Library/esp/esp-idf/components/freertos/port/xtensa/port.c:172

Backtrace:0x40081697:0x3ffda4b0 0x40089b75:0x3ffda4d0 0x40090736:0x3ffda4f0 0x4008d02b:0x3ffda560
0x40081697: panic_abort at /Users/giesel/Library/esp/esp-idf/components/esp_system/panic.c:393

0x40089b75: esp_system_abort at /Users/giesel/Library/esp/esp-idf/components/esp_system/system_api.c:112

0x40090736: abort at /Users/giesel/Library/esp/esp-idf/components/newlib/abort.c:46

0x4008d02b: vPortTaskWrapper at /Users/giesel/Library/esp/esp-idf/components/freertos/port/xtensa/port.c:172
majoreffort commented 2 years ago

Thanks for all your effort.

@majoreffort By using your NETCONN branch I cannot connect to my server anymore. Anyhting I can do to fix this? SC: can't connect to remote 192.168.178.51:1704, err -13

I use mDNS autodiscovery which works fine for me. But I just double checked and have the same issue when trying to use a fixed address. So if you're able to use mDNS that should work. Maybe create a separate issue in CarlosDerSeher's repo for this?

As for the buffering: If you don't have a WROVER module with SPI RAM you really should use the fork from CarlosDerSeher, as he did an excellent job reworking the sync implementation to work with longer buffers.

Aragur commented 2 years ago

I use mDNS autodiscovery which works fine for me. But I just double checked and have the same issue when trying to use a fixed address. So if you're able to use mDNS that should work. Maybe create a separate issue in CarlosDerSeher's repo for this?

Thanks for your fast anser. Do you have a guide how to setup mDNS? I cannot find any documentation on this topic.

CarlosDerSeher commented 2 years ago

@majoreffort

Not sure if I should make a pull request to this repo or your fork @CarlosDerSeher ?

to me it seems like @jorgenkraghjakobsen has abandoned the project, also there is quite some divergence of the repos. You are welcome doing a pull request to my repo, though it shouldn't be a problem to do the same for this repo too as the custom board component is the same for both repositories. You probably will have to make a separte checkout of jorgen's repo and add your changes and do a pull request just for these.

@Aragur mDNS should be activated at the snapserver side and is the default (as far as I know). Also you really should use my fork if you need syncing between clients to work.

Aragur commented 2 years ago

@CarlosDerSeher thanks a lot I now got it working. But is Opus currently not supported on your branch im getting this error:

I (00:02:44.132) SC: Lookup snapcast service on network
I (00:02:47.222) SC: Found 192.168.178.88:1704
I (00:02:47.227) SC: netconn connected
I (00:02:47.229) SC: netconn sent hello message
I (00:02:47.234) SC: Buffer length:  1000
I (00:02:47.234) SC: Latency:        0
I (00:02:47.235) SC: Mute:           0
I (00:02:47.239) SC: Setting volume: 100
I (00:02:47.244) SC: OPUS not implemented yet
Guru Meditation Error: Core  1 panic'ed (IllegalInstruction). Exception was unhandled.
Memory dump at 0x400d7740: b469a51a 1df50406 000000f0
0x400d7740: http_get_task at /Users/giesel/Projects/Private/KiSpi/snapclient-fork2/main/main.c:585

Core  1 register dump:
PC      : 0x400d7747  PS      : 0x00060130  A0      : 0x00000000  A1      : 0x3ffcb8d0
0x400d7747: http_get_task at /Users/giesel/Projects/Private/KiSpi/snapclient-fork2/main/main.c:2682

A2      : 0x3ffb4520  A3      : 0x00000000  A4      : 0x3ffb4518  A5      : 0x3ffb451c
A6      : 0x00000004  A7      : 0x3f401f38  A8      : 0x800d5e01  A9      : 0x3ffcb880
A10     : 0x00000003  A11     : 0x3f401f38  A12     : 0x3f4026b4  A13     : 0x3ffb3f20
A14     : 0x3f401f38  A15     : 0x3ffcb890  SAR     : 0x00000004  EXCCAUSE: 0x00000000
EXCVADDR: 0x00000000  LBEG    : 0x400014fd  LEND    : 0x4000150d  LCOUNT  : 0xffffffff

Backtrace:0x400d7744:0x3ffcb8d0
0x400d7744: http_get_task at /Users/giesel/Projects/Private/KiSpi/snapclient-fork2/main/main.c:588

ELF file SHA256: 3dceee4d40b274fa

If Opus is not implemented yet what codec should be used than? Because PCM uses lot's of the bandwith. (@majoreffort is opus working on your side?)

Using ogg or opus results in an error. Using flac or pcm playback is working with some crackling noises and lot's of errors printed to the console: E (01:41:37.683) PLAYER: Failed to allocate IRAM memory for pcm chunk payload

CarlosDerSeher commented 2 years ago

@Aragur As the log says OPUS is not implemented yet. Use flac instead. Also if you are using NETCONN branch you should lower the buffer length on the server to something below 750ms. Also see https://github.com/CarlosDerSeher/snapclient/issues/2#issuecomment-974759877_

Ah and I just remembered now that I did another modification to flac to reduce overall RAM usage, in file components/flac/flac/src/libFLAC/bitreader.c I set FLACBITREADER_DEFAULT_CAPACITY from static const uint32_t FLACBITREADER_DEFAULT_CAPACITY = 65536u / FLACBITS_PER_WORD; to static const uint32_t FLACBITREADER_DEFAULT_CAPACITY = (65536u / 4) / FLAC__BITS_PER_WORD; Maybe this will help with out of sync too?

You'll have to do that too as I didn't have time to update the documentation or add a patch as I don't know how to integrate it automatically for the flac submodule. Maybe one of you guys have an idea on how to do that?

Aragur commented 2 years ago

@CarlosDerSeher Thanks for your answer. Okay, so that was what I was reading but I thought this was a bug, that opus should be implemented :) So I tested it by ignoring the logs for about 2 hours and I had with 1000ms Buffer and without the modification no syncing issues. Two questions:

CarlosDerSeher commented 2 years ago

If the buffer is too long this can result in skipped frames and isn't handled yet. So you will eventually get out of sync. Errors like E (01:41:37.683) PLAYER: Failed to allocate IRAM memory for pcm chunk payload indicate lost frames which could be audible as crackling noise. You really should reduce the buffer length and try find the best value for your wifi situation where these errors completely go away. Reducing the buffer length has no effect on quality, as long as there are not too much retransmissions for the tcpip connection on your wifi channel. For me 500ms were more than enough.

Reducing FLAC__BITREADER_DEFAULT_CAPACITY has no effect on audio quality but maximum RAM usage. It just means flac's callbacks need to be called more often to get data which results in higher CPU load of the esp32

As a side note: If you have spi ram on your module you could also try use the master branch. I think opus is implemented there though I didn't do any work there for quite some time and don't know how stable it is. Another user reported in #9 it works though

Aragur commented 2 years ago

@CarlosDerSeher I don't have a module with spi ram (only the WROM module). Currently without the flac modification my chip get's pretty hot. Isn't a higher CPU load heating the chip up even more?

CarlosDerSeher commented 2 years ago

what do you mean by pretty hot? I never had problems with high temperature can you still touch the module then you are definitely ok

Aragur commented 2 years ago

what do you mean by pretty hot? I never had problems with high temperature can you still touch the module then you are definitely ok

Okay so I measured it. The chip is not getting over 40°C so everything should be fine :)