Problem with PCM Speaker sounds on hfp_hf example (IDFGH-13304)

espressif / esp-idf

Espressif IoT Development Framework. Official development framework for Espressif SoCs.

Apache License 2.0

13.68k stars 7.29k forks source link

Problem with PCM Speaker sounds on hfp_hf example (IDFGH-13304) #14231

Open mateustoin opened 3 months ago

mateustoin commented 3 months ago

Answers checklist.

[X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
[X] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

General issue report

Hello,

I've been trying for a while to make my speaker work with hfp_hf client example, but it was only possible to hear noises.

[My setup]

ESP-IDF 5.2
Development Kit M5StickC Plus2 (docs)
Speaker Hat-SPK2 PCM Class D Amplifier (docs)

[Description] From the example we can configure the gpio connected to the speaker through _gpio_pcmconfig.h. Using the Speaker pins connected to the devkit as reference, I changed the example pins for the pins the speaker is using:

#define GPIO_OUTPUT_PCM_FSYNC      (0)
#define GPIO_OUTPUT_PCM_CLK_OUT    (26)
#define GPIO_OUTPUT_PCM_DOUT       (25)
#define GPIO_INPUT_PCM_DIN         (35)

The results based on this configuration is that we can hear noises at the opening of sco from HFP call. Since inband ring is enabled, we can hear the noises from the moment the phone is ringing until the call is ended. This indicates that the audio information is going through the pins, but some configuration/pins are wrong.

From the example we can see this information when we're using PCM:

... you should link these GPIO pins to a speaker via I2S port. ...

Is there any example showing how to link the GPIO to a speaker via I2S port? I've also tried to make this configuration, but all of the tentatives lead me to not hearing no sound/noises at all. It would be good if someone could indicate a example for this case or some help on the PCM pins configuration.

The speaker works fine with I2S audio files, but it would be very good to make this work with HFP_HF Client example as well. Right now this is blocking some feature tests in some projects.

Please tell me if you need more information or specific logs in order to solve this issue. Thanks!

esp-qing commented 3 months ago

I've been trying for a while to make my speaker work with hfp_hf client example, but it was only possible to hear noises.

Hi @mateustoin , Can you check the output status of the audio state in the terminal during the hfp call? Is it connected or connected_msbc? or

Only CVSD codec can be used in PCM datapath, and MSBC is not supported for the time being. The correct audio state should be connected(the first figure)

Is there any example showing how to link the GPIO to a speaker via I2S port? I've also tried to make this configuration.......

I think the pin link may be correct. I don't have your hardware environment, but I connected an external speaker to the esp32, and there was only noise during hfp call. After changed the codec mode to CVSD, and hear clear. (I will fix this issue later.)

So please check the audio state first. (If the audio state is connected_msbc, the most direct modification method is to set the value = 1 in handler_callback(value). And check whether the modification is effective, looking forward to your feedback.)

mateustoin commented 3 months ago

Hi @esp-qing, thanks for the quick answer on the issue.

At first, I was able to make the speaker work with different changes from what you suggested, but there is still a few problems to clarify/solve. Let me explain:

Speaker

In the hfp_hf client example I was using the PCM data path by default (that should be using CVSD codec), but after checking the logs you asked, it was reporting --audio state connected_msbc.

[What you suggested/What I've done]

Your suggestion: value = 1 in handler_callback(value)

I dindn't find the correct callback to change the value you mentioned, so what I've done to force the --audio state connected was changing the AT Command responsible for this negotiation: From HCI logs we can see that usually it sends buf = "AT+BAC=1,2\r, then it chooses msbc. With this value equal to 1, it chooses CVSD. After doing this, I was able to hear the audio on the speaker (quality not so good, but It's the speaker's fault).

[The problem] In this case, it shouldn't be selecting CVSD without any more changes? From menuconfig we can see that Wide Band Speech is disabled, and if this is disabled, it should use CVSD. Furthermore, I was not able to find any other ways to make the correct configuration for the example to not use mSbc. Please could you explain about this behavior?

Some other questions

Microphone

I'm also trying to make a Microphone work with this example (microphone model). It's a PDM Microphone, with CLK and DATA pins (simple at first). The hf example shows a configuration of the input pin:

define GPIO_INPUT_PCM_DIN (X)

I've already changed to the correct data pin, but it's not working. Maybe we need to also configure the clock to make this work? I've tried to configure the clock pin as well, but it didn't work.

PCM Configuration

Regarding the PCM Audio Data Path, it's possible to change some audio parameters in this scenario? Like sample rate, etc. Just by curiosity.

Thanks in advance for your help.

esp-qing commented 3 months ago

Hi, @mateustoin Sorry for the late reply.

For the problem of Speaker:

I dindn't find the correct callback to change the value you mentioned, ......

Thehandler_callback(value) is in the bta_hf_client_parse_uint32 function of bta_hf_client_at.c. However, your modification is also feasible.

[The problem] In this case, it shouldn't be selecting CVSD without any more changes? ...... From menuconfig we can see that Wide Band Speech is disabled, and if this is disabled, it should use CVSD. Furthermore, I was not able to find any other ways to make the correct configuration for the example to not use mSbc. Please could you explain about this behavior?

Currently if you choose PCM for datapath, the Wide Band Speech is disabled and It should use CVSD by default. But there are some bugs, which will be fixed later.

For Some other questions of Microphone:

Some other questions Microphone.......

Generally speaking, there is no problem if two clock signal and one data signal are connected correctly. The input and output are good when I test. I can't see any problem for now. You can check whether the connection of these signal is correct and whether the pin configuration is correct.https://docs.espressif.com/projects/esp-idf/zh_CN/latest/esp32/api-reference/peripherals/gpio.html?highlight=gpio
For Some other questions of PCM Configuration:

PCM Configuration......

The latest version of esp-idf master branch provides some PCM configurations in menuconfig, but the PCM sample rate cannot be modified for the time being.

mateustoin commented 2 months ago

Hi @esp-qing,

For now this workaround on HCI/PCM path is working, but do you have an estimated time for the solution? Thank you for the support, is helping a lot. We're developing an application that later will use PCM and HCI, depending on what the user wants, but for now is just PCM. Regarding that, I'm trying to get some kind of audio feedback through the code, similar to what we have in HCI, with the audio callback, but since PCM doesn't have that, I'm trying to figure this out. I think of 2 possibilities:

Get this audio feedback (just the information that the PCM pins has audio) direct with the internal pins, but I don't know if it would be possible to use the ADC reading for this
Configure a I2S channel to read the PCM pins, but I think the configuration of the I2S gpio kind of "cancel" the PCM pins configuration, making the audio muted

Do you have any suggestions regarding this feature? It would be possible to do this?

Thanks in advance for your help.

esp-qing commented 2 months ago

Hi, @mateustoin

For now this workaround on HCI/PCM path is working, but do you have an estimated time for the solution?

This bug has been fixed and is expected to be merged into esp-idf before next Friday.

Regarding that, I'm trying to get some kind of audio feedback through the code, similar to what we have in HCI, with the audio callback, but since PCM doesn't have that, I'm trying to figure this out. I think of 2 possibilities:

The IIS configuration of esp-idf is not effective for the PCM of HFP. The PCM of HFP is equivalent to an independent IIS channel implemented separately. Therefore, your second idea is not feasible. Maybe you can change the pin configuration (in gpio_pcm_config.c) or hardware connection (connect DIN and DOUT) to achieve callback as your first thought.

jtwaleson commented 3 weeks ago

Hi @esp-qing . Thanks for the pointers, managed to get audio working with a change to the esp-idf code. Which version contains the fix? I'm on v5.3.1 and still needed to force CVSD.

esp-qing commented 3 weeks ago

Hi @esp-qing . Thanks for the pointers, managed to get audio working with a change to the esp-idf code. Which version contains the fix? I'm on v5.3.1 and still needed to force CVSD.

Hi, @jtwaleson Release/v5.0 & v5.1 & v5.2 & v5.3 & master all contain the fix. The commit id corresponding to v5.3 is 3af62b6e4868f385e2f33a90af1f46eb604d8ac2 and the master is cd05eea6852ca7e8dd3b40f4239437d9eebefb0b . Please update and try again. If you have any questions, please feel free to ask.

jtwaleson commented 2 weeks ago

Hi @esp-qing . Thanks for the pointers, managed to get audio working with a change to the esp-idf code. Which version contains the fix? I'm on v5.3.1 and still needed to force CVSD.

Hi, @jtwaleson Release/v5.0 & v5.1 & v5.2 & v5.3 & master all contain the fix. The commit id corresponding to v5.3 is 3af62b6 and the master is cd05eea . Please update and try again. If you have any questions, please feel free to ask.

Ah thank you, I was on v5.3.1 ( https://github.com/espressif/esp-idf/commit/c8fc5f643b7a7b0d3b182d3df610844e3dc9bd74 ), will check the correct version. Assuming this is OK now. Thanks!