search

gnumpi / esphome_audio

Custom audio components for ESPHome
Other
54 stars 22 forks source link

Adf_pipeline not initialized #41

Closed xiasi0 closed 3 months ago

xiasi0 commented 4 months ago

I use Snowboy to wake up words and can only complete one voice operation. Then it will prompt me that adf_pipeline is not initialized

Excuse me, is it not supported？

xiasi0 commented 4 months ago

I tested micro_wake_word and found the same situation. Can only be completed once. This is the log: [15:04:26][D][media_player:061]: 's3-dev_media_player' - Setting [15:04:26][D][media_player:068]: Media URL: http://192.168.3.242:8123/api/tts_proxy/2bea3af15933f57a7d0e53bf47780474ebef10f5_zh-cn_6c2e43c6c1_edge_tts.mp3 [15:04:26][D][media_player:074]: Announcement: yes [15:04:26][D][adf_media_player:030]: Got control call in state 1 [15:04:26][D][esp_adf_pipeline:050]: Starting request, current state UNINITIALIZED [15:04:26][V][esp-idf:000]: I (25310) MP3_DECODER: MP3 init

[15:04:26][V][esp-idf:000]: I (25322) I2S: DMA Malloc info, datalen=blocksize=2048, dma_buf_count=4

[15:04:26][D][i2s_audio:072]: Installing driver : yes [15:04:26][D][esp_adf_pipeline:358]: pipeline tag 0, http [15:04:26][D][esp_adf_pipeline:358]: pipeline tag 1, decoder [15:04:26][D][esp_adf_pipeline:358]: pipeline tag 2, i2s_out [15:04:26][V][esp-idf:000]: I (25343) AUDIO_PIPELINE: link el->rb, el:0x3d81c0f8, tag:http, rb:0x3d81c64c

[15:04:26][V][esp-idf:000]: I (25351) AUDIO_PIPELINE: link el->rb, el:0x3d81c2e8, tag:decoder, rb:0x3d81d68c

[15:04:26][D][esp_adf_pipeline:370]: Setting up event listener. [15:04:26][D][esp_adf_pipeline:302]: State changed from UNINITIALIZED to PREPARING [15:04:26][I][adf_media_player:135]: got new pipeline state: 1 [15:04:26][D][adf_i2s_out:127]: Set final i2s settings: 16000 [15:04:26][W][component:237]: Component voice_assistant took a long time for an operation (124 ms). [15:04:26][W][component:238]: Components should block for at most 30 ms. [15:04:26][D][voice_assistant:625]: Event Type: 2 [15:04:26][D][voice_assistant:715]: Assist Pipeline ended [15:04:26][V][esp-idf:000]: I (25423) AUDIO_THREAD: The http task allocate stack on external memory

[15:04:26][V][esp-idf:000]: I (25426) AUDIO_ELEMENT: [http-0x3d81c0f8] Element task created

[15:04:26][V][esp-idf:000]: I (25433) AUDIO_THREAD: The decoder task allocate stack on external memory

[15:04:26][V][esp-idf:000]: I (25443) AUDIO_ELEMENT: [decoder-0x3d81c2e8] Element task created

[15:04:26][V][esp-idf:000][http]: I (25453) AUDIO_ELEMENT: [http] AEL_MSG_CMD_RESUME,state:1

[15:04:26][V][esp-idf:000][decoder]: I (25464) AUDIO_ELEMENT: [decoder] AEL_MSG_CMD_RESUME,state:1

[15:04:26][D][esp_aud:000][decoStreamer status: 2 [15:04:26][D][esp_audio_sources:098]: decoder status: 2 [15:04:26][W][component:237]: Component adf_pipeline.media_player took a long time for an operation (61 ms). [15:04:26][W][component:237]: Component adf_pipeline.media_player took a long time for an operation (61 ms). [15:04:26][W][component:238]: Components should block for at most 30 ms. [15:04:26][I][HTTPStreamReader:129]: [ * ] Receive music info from mp3 decoder, sample_rates=24000, bits=16, ch=1 [15:04:26][D][adf_i2s_out:114]: update i2s clk settings: rate:24000 bits:16 ch:1 [15:04:26][V][esp-idf:000]: I (25519) I2S: DMA Malloc info, datalen=blocksize=1024, dma_buf_count=4

[15:04:26][D][adf_i2s_out:127]: Set final i2s settings: 24000 [15:04:26][V][esp-idf:000][decoder]: W (25536) AUDIO_ELEMENT: OUT-[decoder] AEL_IO_ABORT

[15:04:26][V][esp-idf:000][decoder]: W (25546) MP3_DECODER: output aborted -3

[15:04:26][V][esp-idf:000][decoder]: I (25557) MP3_DECODER: Closed

[15:04:26][D][esp_adf_pipeline:302]: State changed from PREPARING to STARTING [15:04:26][I][adf_media_player:135]: got new pipeline state: 2 [15:04:26][D][adf_i2s_out:127]: Set final i2s settings: 24000 [15:04:26][V][esp-idf:000]: I (25588) AUDIO_ELEMENT: [i2s_out-0x3d81c4b4] Element task created

[15:04:26][V][esp-idf:000]: I (25597) AUDIO_PIPELINE: Func:audio_pipeline_run, Line:359, MEM Total:8424887 Bytes, Inter:174452 Bytes, Dram:174452 Bytes

[15:04:26][V][esp-idf:000][http]: I (25607) AUDIO_ELEMENT: [http] AEL_MSG_CMD_RESUME,state:1

[15:04:26][V][esp-idf:000][decoder]: I (25618) AUDIO_ELEMENT: [decoder] AEL_MSG_CMD_RESUME,state:1

[15:04:26][V][esp-idf:000][i2s_out]: I (25628) AUDIO_ELEMENT: [i2s_out] AEL_MSG_CMD_RESUME,state:1

[15:04:26][V][esp-idf:000][i2s_out]: I (25639) I2S_STREAM: AUDIO_STREAM_WRITER

[15:04:26][I][esp_adf_pipeline:214]: [ i2s_out ] status: 12 [15:04:26][D][esp_adf_pipeline:131]: Check element [http] status, 3 [15:04:26][D][esp_adf_pipeline:131]: Check element [decoder] status, 2 [15:04:27][I][esp_adf_pipeline:214]: [ decoder ] status: 12 [15:04:27][D][esp_adf_pipeline:131]: Check element [http] status, 3 [15:04:27][D][esp_adf_pipeline:131]: Check element [decoder] status, 3 [15:04:27][D][esp_adf_pipeline:131]: Check element [i2s_out] status, 3 [15:04:27][D][esp_adf_pipeline:302]: State changed from STARTING to RUNNING [15:04:27][I][adf_media_player:135]: got new pipeline state: 3 [15:04:27][D][adf_i2s_out:127]: Set final i2s settings: 24000 [15:04:27][I][HTTPStreamReader:129]: [ * ] Receive music info from mp3 decoder, sample_rates=24000, bits=16, ch=1 [15:04:27][D][adf_i2s_out:127]: Set final i2s settings: 24000 [15:04:27][V][esp-idf:000][http]: W (26660) HTTP_STREAM: No more data,errno:0, total_bytes:14221, rlen = 0

[15:04:27][V][esp-idf:000][http]: I (26663) AUDIO_ELEMENT: IN-[http] AEL_IO_DONE,0

[15:04:27][I][esp_adf_pipeline:214]: [ http ] status: 15 [15:04:27][D][esp_adf_pipeline:302]: State changed from RUNNING to STOPPING [15:04:27][I][adf_media_player:135]: got new pipeline state: 4 [15:04:28][V][esp-idf:000][decoder]: I (27363) AUDIO_ELEMENT: IN-[decoder] AEL_IO_DONE,-2

[15:04:29][V][esp-idf:000][decoder]: I (27723) MP3_DECODER: Closed

[15:04:29][V][esp-idf:000][i2s_out]: I (27829) AUDIO_ELEMENT: IN-[i2s_out] AEL_IO_DONE,-2

[15:04:29][D][esp_adf_pipeline:400]: Called deinit_all [15:04:29][V][esp-idf:000]: I (28003) AUDIO_PIPELINE: audio_pipeline_unlinked

[15:04:29][V][esp-idf:000]: W (28005) AUDIO_ELEMENT: [http] Element has not create when AUDIO_ELEMENT_TERMINATE

[15:04:29][V][esp-idf:000]: W (28008) AUDIO_ELEMENT: [decoder] Element has not create when AUDIO_ELEMENT_TERMINATE

[15:04:29][V][esp-idf:000]: W (28010) AUDIO_ELEMENT: [i2s_out] Element has not create when AUDIO_ELEMENT_TERMINATE

[15:04:29][V][esp-idf:000]: I (28025) I2S: DMA queue destroyed

[15:04:29][D][esp_adf_pipeline:302]: State changed from STOPPING to UNINITIALIZED [15:04:29][I][adf_media_player:135]: got new pipeline state: 0

gnumpi commented 4 months ago

Can you try using the dev-next branch?

external_components:
  - source:
      type: git
      url: https://github.com/gnumpi/esphome_audio
      ref: dev-next
    components: [ adf_pipeline, i2s_audio]

If you are using it already, can you share your config?

xiasi0 commented 4 months ago

I tried this branch, this is my configuration：

 voice_assist_idle_phase_id: '1'
 voice_assist_listening_phase_id: '2'
 voice_assist_thinking_phase_id: '3'
 voice_assist_replying_phase_id: '4'
 voice_assist_not_ready_phase_id: '10'
 voice_assist_error_phase_id: '11'  
 voice_assist_muted_phase_id: '12'
 #pins
 i2s_out_lrclk_pin: GPIO11 # LRC on Max98357
 i2s_out_bclk_pin: GPIO9 # BCLK on Max98357
 i2s_in_lrclk_pin: GPIO3 # WS on INMP441
 i2s_in_bclk_pin: GPIO2 # SLK on INMP441
 speaker_pin: GPIO8 # DIN on Max98357
 mic_pin: GPIO4 # SD on INMP441
esphome:
 name: esp32s3
 friendly_name: ESP32S3
 platformio_options:
   board_build.flash_mode: dio
psram:
 mode:  octal
 speed: 80MHz

esp32:
 board: esp32-s3-devkitc-1
 variant: ESP32S3
 framework:
   type: esp-idf
   version: recommended
   sdkconfig_options:
     # need to set a s3 compatible board for the adf-sdk to compile
     # board specific code is not used though
     CONFIG_ESP32_S3_BOX_BOARD: "y"
     CONFIG_ESP32_WIFI_STATIC_RX_BUFFER_NUM: "16"
     CONFIG_ESP32_WIFI_DYNAMIC_RX_BUFFER_NUM: "512"
     CONFIG_TCPIP_RECVMBOX_SIZE: "512"

     CONFIG_TCP_SND_BUF_DEFAULT: "65535"
     CONFIG_TCP_WND_DEFAULT: "512000"
     CONFIG_TCP_RECVMBOX_SIZE: "512"

     CONFIG_ESP32S3_DEFAULT_CPU_FREQ_240: "y"
     CONFIG_ESP32S3_DATA_CACHE_64KB:      "y"
     CONFIG_ESP32S3_DATA_CACHE_LINE_64B:  "y"
# Enable logging
logger:

# Enable Home Assistant API
api:
 encryption:
   key: "my key"

ota:
 password: "my paw"

wifi:
 ssid: !secret wifi_ssid
 password: !secret wifi_password

 # Enable fallback hotspot (captive portal) in case wifi connection fails
 ap:
   ssid: "Esp32S3 Fallback Hotspot"
   password: "ZERO9EbKbp7c"
external_components:
 - source:
     type: git
     url: https://github.com/gnumpi/esphome_audio
     ref: dev-next
   components: [ adf_pipeline, i2s_audio]

adf_pipeline:
 - platform: i2s_audio
   type: audio_out
   id: adf_i2s_out
   i2s_audio_id: i2s_out
   i2s_dout_pin: ${speaker_pin}

 - platform: i2s_audio
   type: audio_in
   id: adf_i2s_in
   i2s_audio_id: i2s_in
   i2s_din_pin: ${mic_pin}
   pdm: false
   channel: left
   sample_rate: 16000
   bits_per_sample: 32bit

microphone:
 - platform: adf_pipeline
   id: adf_microphone
   gain_log2: 3
   keep_pipeline_alive: false
   pipeline:
     - adf_i2s_in
     - self

media_player:
 - platform: adf_pipeline
   id: adf_media_player
   name: s3-dev_media_player
   keep_pipeline_alive: false
   internal: false
   pipeline:
     - self
     - adf_i2s_out

# This is our two i2s buses with the correct pins. 
# You can refer to the wirinng diagram of our voice assistant for more details 
i2s_audio:
 - id: i2s_out
   i2s_lrclk_pin: ${i2s_out_lrclk_pin}
   i2s_bclk_pin: ${i2s_out_bclk_pin}
 - id: i2s_in
   i2s_lrclk_pin: ${i2s_in_lrclk_pin}
   i2s_bclk_pin: ${i2s_in_bclk_pin}

# This is the declaration of our voice assistant
# It references the microphone and speaker declared above.
voice_assistant:
 id: va
 microphone: adf_microphone
 media_player: adf_media_player
 use_wake_word: true
 noise_suppression_level: 4 #4
 auto_gain: 31dBFS # 31dBFS
 volume_multiplier: 8 # 8.0

binary_sensor:
 - platform: status
   name: API Connection
   id: api_connection
   filters:
     - delayed_on: 1s
   on_press:
     - if:
         condition:
           switch.is_on: use_wake_word
         then:
           - voice_assistant.start_continuous:
   on_release:
     - if:
         condition:
           switch.is_on: use_wake_word
         then:
           - voice_assistant.stop:

switch:
 - platform: template
   name: Use wake word
   id: use_wake_word
   optimistic: true
   restore_mode: RESTORE_DEFAULT_ON
   entity_category: config
   on_turn_on:
     - lambda: id(va).set_use_wake_word(true);
     - if:
         condition:
           not:
             - voice_assistant.is_running
         then:
           - voice_assistant.start_continuous

   on_turn_off:
     - voice_assistant.stop
     - lambda: id(va).set_use_wake_word(false); ```

This is an error that has occurred, please help me。After completing one task,[esp_adf_pipeline:302]: State changed from STOPPING to UNINITIALIZED


[19:52:18][D][voice_assistant:502]: State changed from IDLE to START_MICROPHONE
[19:52:18][D][voice_assistant:508]: Desired state set to START_PIPELINE
[19:52:18][D][voice_assistant:220]: Starting Microphone
[19:52:19][D][ring_buffer:024]: Created ring buffer with size 16384
[19:52:19][D][esp_adf_pipeline:050]: Starting request, current state UNINITIALIZED
[19:52:19][D][esp-idf:000]: I (7197) I2S: DMA Malloc info, datalen=blocksize=1024, dma_buf_count=4

[19:52:19][D][i2s_audio:072]: Installing driver : yes
[19:52:19][D][esp_adf_pipeline:358]: pipeline tag 0, i2s_in
[19:52:19][D][esp_adf_pipeline:358]: pipeline tag 1, pcm_reader
[19:52:19][D][esp-idf:000]: I (7206) AUDIO_PIPELINE: link el->rb, el:0x3d80527c, tag:i2s_in, rb:0x3d805558

[19:52:19][D][esp_adf_pipeline:370]: Setting up event listener.
[19:52:19][D][esp_adf_pipeline:302]: State changed from UNINITIALIZED to PREPARING
[19:52:19][D][voice_assistant:502]: State changed from START_MICROPHONE to STARTING_MICROPHONE
[19:52:19][D][esp_audio_sinks:053]: Set bitdepth to 32
[19:52:19][D][esp_adf_pipeline:302]: State changed from PREPARING to STARTING
[19:52:19][D][esp-idf:000]: I (7220) AUDIO_ELEMENT: [i2s_in-0x3d80527c] Element task created

[19:52:19][D][esp-idf:000]: I (7223) AUDIO_ELEMENT: [pcm_reader-0x3d805410] Element task created

[19:52:19][D][esp-idf:000]: I (7226) AUDIO_PIPELINE: Func:audio_pipeline_run, Line:359, MEM Total:8511975 Bytes, Inter:147912 Bytes, Dram:147912 Bytes

[19:52:19][D][esp-idf:000][i2s_in]: I (7229) AUDIO_ELEMENT: [i2s_in] AEL_MSG_CMD_RESUME,state:1

[19:52:19][D][esp-idf:000]: I (7232) AUDIO_PIPELINE: Pipeline started

[19:52:19][I][esp_adf_pipeline:214]: [ pcm_reader ] status: 12
[19:52:19][D][esp_adf_pipeline:131]: Check element [i2s_in] status, 3
[19:52:19][D][esp_adf_pipeline:131]: Check element [pcm_reader] status, 3
[19:52:19][D][esp_adf_pipeline:302]: State changed from STARTING to RUNNING
[19:52:19][D][voice_assistant:502]: State changed from STARTING_MICROPHONE to START_PIPELINE
[19:52:19][I][esp_adf_pipeline:214]: [ pcm_reader ] status: 12
[19:52:19][D][voice_assistant:274]: Requesting start...
[19:52:19][D][voice_assistant:502]: State changed from START_PIPELINE to STARTING_PIPELINE
[19:52:19][I][esp_adf_pipeline:214]: [ i2s_in ] status: 12
[19:52:19][D][voice_assistant:523]: Client started, streaming microphone
[19:52:19][D][voice_assistant:502]: State changed from STARTING_PIPELINE to STREAMING_MICROPHONE
[19:52:19][D][voice_assistant:508]: Desired state set to STREAMING_MICROPHONE
[19:52:19][D][voice_assistant:625]: Event Type: 1
[19:52:19][D][voice_assistant:628]: Assist Pipeline running
[19:52:19][D][voice_assistant:625]: Event Type: 9
[19:52:19][D][voice_assistant:625]: Event Type: 0
[19:52:19][E][voice_assistant:755]: Error: no_wake_word - No wake word detected
[19:52:19][D][voice_assistant:618]: Signaling stop...
[19:52:19][D][voice_assistant:502]: State changed from STREAMING_MICROPHONE to STOP_MICROPHONE
[19:52:19][D][voice_assistant:508]: Desired state set to IDLE
[19:52:19][D][esp_adf_pipeline:302]: State changed from RUNNING to STOPPING
[19:52:19][D][voice_assistant:502]: State changed from STOP_MICROPHONE to STOPPING_MICROPHONE
[19:52:19][D][voice_assistant:625]: Event Type: 2
[19:52:19][D][voice_assistant:715]: Assist Pipeline ended
[19:52:19][D][esp-idf:000][i2s_in]: W (7359) AUDIO_ELEMENT: OUT-[i2s_in] AEL_IO_ABORT

[19:52:19][D][esp-idf:000][i2s_in]: W (7362) AUDIO_ELEMENT: OUT-[i2s_in] AEL_IO_ABORT

[19:52:19][D][esp-idf:000][i2s_in]: W (7366) AUDIO_ELEMENT: OUT-[i2s_in] AEL_IO_ABORT

[19:52:19][D][esp-idf:000][i2s_in]: W (7369) AUDIO_ELEMENT: OUT-[i2s_in] AEL_IO_ABORT

[19:52:19][D][esp_adf_pipeline:400]: Called deinit_all
[19:52:19][D][esp-idf:000]: I (7377) AUDIO_PIPELINE: audio_pipeline_unlinked

[19:52:19][D][esp-idf:000]: W (7379) AUDIO_ELEMENT: [i2s_in] Element has not create when AUDIO_ELEMENT_TERMINATE

[19:52:19][D][esp-idf:000]: W (7382) AUDIO_ELEMENT: [pcm_reader] Element has not create when AUDIO_ELEMENT_TERMINATE

[19:52:19][D][esp-idf:000]: I (7384) I2S: DMA queue destroyed

[19:52:19][D][esp_adf_pipeline:302]: State changed from STOPPING to UNINITIALIZED
[19:52:19][D][voice_assistant:502]: State changed from STOPPING_MICROPHONE to IDLE
[19:52:19][D][voice_assistant:502]: State changed from IDLE to START_MICROPHONE
[19:52:19][D][voice_assistant:508]: Desired state set to START_PIPELINE
[19:52:19][D][voice_assistant:220]: Starting Microphone
[19:52:19][D][esp_adf_pipeline:050]: Starting request, current state UNINITIALIZED
[19:52:19][D][esp-idf:000]: I (7400) I2S: DMA Malloc info, datalen=blocksize=1024, dma_buf_count=4

[19:52:19][D][i2s_audio:072]: Installing driver : yes
[19:52:19][D][esp_adf_pipeline:358]: pipeline tag 0, i2s_in
[19:52:19][D][esp_adf_pipeline:358]: pipeline tag 1, pcm_reader
[19:52:19][D][esp-idf:000]: I (7410) AUDIO_PIPELINE: link el->rb, el:0x3d80527c, tag:i2s_in, rb:0x3d805558

[19:52:19][D][esp_adf_pipeline:370]: Setting up event listener.
[19:52:19][D][esp_adf_pipeline:302]: State changed from UNINITIALIZED to PREPARING
[19:52:19][D][voice_assistant:502]: State changed from START_MICROPHONE to STARTING_MICROPHONE
[19:52:19][D][esp_adf_pipeline:302]: State changed from PREPARING to STARTING
[19:52:19][D][esp-idf:000]: I (7422) AUDIO_ELEMENT: [i2s_in-0x3d80527c] Element task created

[19:52:19][D][esp-idf:000]: I (7425) AUDIO_ELEMENT: [pcm_reader-0x3d805410] Element task created

[19:52:19][D][esp-idf:000]: I (7427) AUDIO_PIPELINE: Func:audio_pipeline_run, Line:359, MEM Total:8497771 Bytes, Inter:133708 Bytes, Dram:133708 Bytes

[19:52:19][D][esp-idf:000][i2s_in]: I (7429) AUDIO_ELEMENT: [i2s_in] AEL_MSG_CMD_RESUME,state:1

[19:52:19][D][esp-idf:000]: I (7432) AUDIO_PIPELINE: Pipeline started

[19:52:19][I][esp_adf_pipeline:214]: [ pcm_reader ] status: 12
[19:52:19][D][esp_adf_pipeline:131]: Check element [i2s_in] status, 3
[19:52:19][D][esp_adf_pipeline:131]: Check element [pcm_reader] status, 3
[19:52:19][D][esp_adf_pipeline:302]: State changed from STARTING to RUNNING
[19:52:19][D][voice_assistant:502]: State changed from STARTING_MICROPHONE to START_PIPELINE
[19:52:19][I][esp_adf_pipeline:214]: [ pcm_reader ] status: 12
[19:52:19][D][voice_assistant:274]: Requesting start...
[19:52:19][D][voice_assistant:502]: State changed from START_PIPELINE to STARTING_PIPELINE
[19:52:19][I][esp_adf_pipeline:214]: [ i2s_in ] status: 12
[19:52:19][D][voice_assistant:523]: Client started, streaming microphone
[19:52:19][D][voice_assistant:502]: State changed from STARTING_PIPELINE to STREAMING_MICROPHONE
[19:52:19][D][voice_assistant:508]: Desired state set to STREAMING_MICROPHONE
[19:52:19][D][voice_assistant:625]: Event Type: 1
[19:52:19][D][voice_assistant:628]: Assist Pipeline running
[19:52:19][D][voice_assistant:625]: Event Type: 9
[19:52:25][D][voice_assistant:625]: Event Type: 0
[19:52:25][D][voice_assistant:625]: Event Type: 2
[19:52:25][D][voice_assistant:715]: Assist Pipeline ended
[19:52:25][D][voice_assistant:502]: State changed from STREAMING_MICROPHONE to IDLE
[19:52:25][D][voice_assistant:508]: Desired state set to IDLE
[19:52:25][D][voice_assistant:502]: State changed from IDLE to START_MICROPHONE
[19:52:25][D][voice_assistant:508]: Desired state set to START_PIPELINE
[19:52:25][D][voice_assistant:220]: Starting Microphone
[19:52:25][D][voice_assistant:502]: State changed from START_MICROPHONE to STARTING_MICROPHONE
[19:52:25][D][voice_assistant:502]: State changed from STARTING_MICROPHONE to START_PIPELINE
[19:52:25][D][voice_assistant:274]: Requesting start...
[19:52:25][D][voice_assistant:502]: State changed from START_PIPELINE to STARTING_PIPELINE
[19:52:25][D][voice_assistant:523]: Client started, streaming microphone
[19:52:25][D][voice_assistant:502]: State changed from STARTING_PIPELINE to STREAMING_MICROPHONE
[19:52:25][D][voice_assistant:508]: Desired state set to STREAMING_MICROPHONE
[19:52:25][D][voice_assistant:625]: Event Type: 1
[19:52:25][D][voice_assistant:628]: Assist Pipeline running
[19:52:25][D][voice_assistant:625]: Event Type: 9
[19:52:31][D][voice_assistant:625]: Event Type: 10
[19:52:31][D][voice_assistant:634]: Wake word detected
[19:52:31][D][voice_assistant:625]: Event Type: 3
[19:52:31][D][voice_assistant:639]: STT started
[19:52:32][D][voice_assistant:625]: Event Type: 11
[19:52:32][D][voice_assistant:779]: Starting STT by VAD
[19:52:34][D][voice_assistant:625]: Event Type: 12
[19:52:34][D][voice_assistant:783]: STT by VAD end
[19:52:34][D][voice_assistant:502]: State changed from STREAMING_MICROPHONE to STOP_MICROPHONE
[19:52:34][D][voice_assistant:508]: Desired state set to AWAITING_RESPONSE
[19:52:34][D][esp_adf_pipeline:302]: State changed from RUNNING to STOPPING
[19:52:34][D][voice_assistant:502]: State changed from STOP_MICROPHONE to STOPPING_MICROPHONE
[19:52:34][D][esp_adf_pipeline:400]: Called deinit_all
[19:52:34][D][esp-idf:000]: I (22590) AUDIO_PIPELINE: audio_pipeline_unlinked

[19:52:34][D][esp-idf:000]: W (22592) AUDIO_ELEMENT: [i2s_in] Element has not create when AUDIO_ELEMENT_TERMINATE

[19:52:34][D][esp-idf:000]: W (22595) AUDIO_ELEMENT: [pcm_reader] Element has not create when AUDIO_ELEMENT_TERMINATE

[19:52:34][D][esp-idf:000]: I (22598) I2S: DMA queue destroyed

[19:52:34][D][esp_adf_pipeline:302]: State changed from STOPPING to UNINITIALIZED
[19:52:34][D][voice_assistant:502]: State changed from STOPPING_MICROPHONE to AWAITING_RESPONSE
[19:52:34][D][voice_assistant:625]: Event Type: 4
[19:52:34][D][voice_assistant:653]: Speech recognised as: "Turn off the children's room headlights"
[19:52:34][D][voice_assistant:625]: Event Type: 5
[19:52:34][D][voice_assistant:658]: Intent started
[19:52:34][D][voice_assistant:625]: Event Type: 6
[19:52:34][D][voice_assistant:625]: Event Type: 7
[19:52:34][D][voice_assistant:681]: Response: "The children's room headlights have been turned off"
[19:52:34][D][voice_assistant:625]: Event Type: 8
[19:52:34][D][voice_assistant:701]: Response URL: "http://192.168.3.242:8123/api/tts_proxy/ec9a8613722f2527f9ca2c7c8308081d21f3b92f_zh-cn_6c2e43c6c1_edge_tts.mp3"
[19:52:34][D][voice_assistant:502]: State changed from AWAITING_RESPONSE to STREAMING_RESPONSE
[19:52:34][D][voice_assistant:508]: Desired state set to STREAMING_RESPONSE
[19:52:34][D][media_player:061]: 's3-dev_media_player' - Setting
[19:52:34][D][media_player:068]:   Media URL: http://192.168.3.242:8123/api/tts_proxy/ec9a8613722f2527f9ca2c7c8308081d21f3b92f_zh-cn_6c2e43c6c1_edge_tts.mp3
[19:52:34][D][media_player:074]:  Announcement: yes
[19:52:34][D][adf_media_player:030]: Got control call in state 1
[19:52:34][D][esp_adf_pipeline:050]: Starting request, current state UNINITIALIZED
[19:52:34][D][esp-idf:000]: I (23168) MP3_DECODER: MP3 init

[19:52:34][D][esp-idf:000]: I (23171) I2S: DMA Malloc info, datalen=blocksize=2048, dma_buf_count=4

[19:52:34][D][i2s_audio:072]: Installing driver : yes
[19:52:34][D][esp_adf_pipeline:358]: pipeline tag 0, http
[19:52:34][D][esp_adf_pipeline:358]: pipeline tag 1, decoder
[19:52:34][D][esp_adf_pipeline:358]: pipeline tag 2, i2s_out
[19:52:34][D][esp-idf:000]: I (23181) AUDIO_PIPELINE: link el->rb, el:0x3d805240, tag:http, rb:0x3d805794

[19:52:34][D][esp-idf:000]: I (23185) AUDIO_PIPELINE: link el->rb, el:0x3d805430, tag:decoder, rb:0x3d8067d4

[19:52:34][D][esp_adf_pipeline:370]: Setting up event listener.
[19:52:34][D][esp_adf_pipeline:302]: State changed from UNINITIALIZED to PREPARING
[19:52:34][I][adf_media_player:135]: got new pipeline state: 1
[19:52:34][D][adf_i2s_out:127]: Set final i2s settings: 16000
[19:52:34][D][voice_assistant:625]: Event Type: 2
[19:52:34][D][voice_assistant:715]: Assist Pipeline ended
[19:52:35][D][esp-idf:000]: I (23206) AUDIO_THREAD: The http task allocate stack on external memory

[19:52:35][D][esp-idf:000]: I (23208) AUDIO_ELEMENT: [http-0x3d805240] Element task created

[19:52:35][D][esp-idf:000]: I (23211) AUDIO_THREAD: The decoder task allocate stack on external memory

[19:52:35][D][esp-idf:000]: I (23213) AUDIO_ELEMENT: [decoder-0x3d805430] Element task created

[19:52:35][D][esp-idf:000][http]: I (23215) AUDIO_ELEMENT: [http] AEL_MSG_CMD_RESUME,state:1

[19:52:35][D][esp-idf:000][decoder]: I (23218) AUDIO_ELEMENT: [decoder] AEL_MSG_CMD_RESUME,state:1

[19:52:35][D][esp-idf:000][decoder]: 
ERROR Fatal error: protocol.data_received() call failed.
protocol: <aioesphomeapi._frame_helper.noise.APINoiseFrameHelper object at 0x7f3ad1c5c040>
transport: <_SelectorSocketTransport fd=7 read=polling write=<idle, bufsize=0>>
Traceback (most recent call last):
  File "/usr/lib/python3.11/asyncio/selector_events.py", line 1009, in _read_ready__data_received
    self._protocol.data_received(data)
  File "aioesphomeapi/_frame_helper/noise.py", line 136, in aioesphomeapi._frame_helper.noise.APINoiseFrameHelper.data_received
  File "aioesphomeapi/_frame_helper/noise.py", line 163, in aioesphomeapi._frame_helper.noise.APINoiseFrameHelper.data_received
  File "aioesphomeapi/_frame_helper/noise.py", line 319, in aioesphomeapi._frame_helper.noise.APINoiseFrameHelper._handle_frame
  File "/usr/local/lib/python3.11/dist-packages/noise/state.py", line 74, in decrypt_with_ad
    plaintext = self.cipher.decrypt(self.k, self.n, ad, ciphertext)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/noise/backends/default/ciphers.py", line 13, in decrypt
    return self.cipher.decrypt(nonce=self.format_nonce(n), data=ciphertext, associated_data=ad)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "src/chacha20poly1305_reuseable/__init__.py", line 127, in chacha20poly1305_reuseable.ChaCha20Poly1305Reusable.decrypt
  File "src/chacha20poly1305_reuseable/__init__.py", line 147, in chacha20poly1305_reuseable.ChaCha20Poly1305Reusable.decrypt
  File "src/chacha20poly1305_reuseable/__init__.py", line 263, in chacha20poly1305_reuseable._decrypt_with_fixed_nonce_len
  File "src/chacha20poly1305_reuseable/__init__.py", line 273, in chacha20poly1305_reuseable._decrypt_data
cryptography.exceptions.InvalidTag
WARNING esp32s3 @ 192.168.3.24: Connection error occurred: esp32s3 @ 192.168.3.24: Invalid encryption key: received_name=esp32s3
INFO Processing unexpected disconnect from ESPHome API for esp32s3 @ 192.168.3.24
WARNING Disconnected from API
INFO Successfully connected to esp32s3 @ 192.168.3.24 in 0.015s
INFO Successful handshake with esp32s3 @ 192.168.3.24 in 0.080s
[19:52:35][D][esp-idf:000][http]: I (23397) HTTP_CLIENT: Body received in fetch header state, 0x3fccd8c7, 1841

[19:52:35][D][esp-idf:000][http]: I (23401) HTTP_STREAM: total_bytes=15377

[19:52:35][I][esp_adf_pipeline:214]: [ http ] status: 12
[19:52:35][D][esp_adf_pipeline:131]: Check element [http] status, 3
[19:52:35][D][esp_adf_pipeline:131]: Check element [decoder] status, 2
[19:52:35][I][esp_adf_pipeline:214]: [ decoder ] status: 12
[19:52:35][D][esp_adf_pipeline:131]: Check element [http] status, 3
[19:52:35][D][esp_adf_pipeline:131]: Check element [decoder] status, 3
[19:52:35][D][esp_adf_pipeline:131]: Check element [i2s_out] status, 3
[19:52:35][D][esp_adf_pipeline:302]: State changed from STARTING to RUNNING
[19:52:35][I][adf_media_player:135]: got new pipeline state: 3
[19:52:35][D][adf_i2s_out:127]: Set final i2s settings: 24000
[19:52:35][I][HTTPStreamReader:129]: [ * ] Receive music info from mp3 decoder, sample_rates=24000, bits=16, ch=1
[19:52:35][D][adf_i2s_out:127]: Set final i2s settings: 24000
[19:52:36][D][esp-idf:000][http]: W (24632) HTTP_STREAM: No more data,errno:0, total_bytes:15377, rlen = 0

[19:52:36][D][esp-idf:000][http]: I (24636) AUDIO_ELEMENT: IN-[http] AEL_IO_DONE,0

[19:52:36][I][esp_adf_pipeline:214]: [ http ] status: 15
[19:52:36][D][esp_adf_pipeline:302]: State changed from RUNNING to STOPPING
[19:52:36][I][adf_media_player:135]: got new pipeline state: 4
[19:52:37][D][esp-idf:000][decoder]: I (25334) AUDIO_ELEMENT: IN-[decoder] AEL_IO_DONE,-2

[19:52:37][D][esp-idf:000][decoder]: I (25695) MP3_DECODER: Closed

[19:52:37][D][esp-idf:000][i2s_out]: I (25802) AUDIO_ELEMENT: IN-[i2s_out] AEL_IO_DONE,-2

[19:52:37][D][esp_adf_pipeline:400]: Called deinit_all
[19:52:37][D][esp-idf:000]: I (25975) AUDIO_PIPELINE: audio_pipeline_unlinked

[19:52:37][D][esp-idf:000]: W (25977) AUDIO_ELEMENT: [http] Element has not create when AUDIO_ELEMENT_TERMINATE

[19:52:37][D][esp-idf:000]: W (25979) AUDIO_ELEMENT: [decoder] Element has not create when AUDIO_ELEMENT_TERMINATE

[19:52:37][D][esp-idf:000]: W (25982) AUDIO_ELEMENT: [i2s_out] Element has not create when AUDIO_ELEMENT_TERMINATE

[19:52:37][D][esp-idf:000]: I (25984) I2S: DMA queue destroyed

[19:52:37][D][esp_adf_pipeline:302]: State changed from STOPPING to UNINITIALIZED
[19:52:37][I][adf_media_player:135]: got new pipeline state: 0
[19:57:12][I][ota:117]: Boot seems successful, resetting boot loop counter.
[19:57:12][D][esp32.preferences:114]: Saving 1 preferences to flash...
[19:57:12][D][esp32.preferences:143]: Saving 1 preferences to flash: 0 cached, 1 written, 0 failed ```
gnumpi commented 4 months ago

esp32s3 @ 192.168.3.24: Invalid encryption key: received_name=esp32s3 INFO Processing unexpected disconnect from ESPHome API for esp32s3 @ 192.168.3.24

Don’t think that this is related to the adf-pipeline

So, do you hear any audio? Wrong speed, or just nothing at all?

xiasi0 commented 3 months ago

esp32s3 @ 192.168.3.24: Invalid encryption key: received_name=esp32s3 INFO Processing unexpected disconnect from ESPHome API for esp32s3 @ 192.168.3.24

Don’t think that this is related to the adf-pipeline

So, do you hear any audio? Wrong speed, or just nothing at all?

Thank you, I saw this error, but I always thought it wasn't an issue with Esphome. I tried all possible methods and finally uninstalled and reinstalled Esphome. It is working normally now. I'm not sure what the reason is. But it is working very well now and the sound is also very clear. Thank you again for your reply

gnumpi commented 3 months ago

Ahh that’s good to hear!!!