ESPHome voice_assistant won't compile when speaker is removed from yaml configuration

The problem

I am trying to port the m5stack.atom-echo-voice-assistant configuration run on my M5StickC Plus ESP32.

The M5Stick has a microphone but no speaker, so I was hoping to be able to use this for the voice_assistant component and just use the buzzer in the M5StickC to indicate a command has been processed. The microphone hardware on this device is exactly the same as on the M5Stack Atom Echo.

In the ESPHome documentation it says the speaker component is optional for voice_assistant.

When I remove the "speaker" option from the config I get a compilation error and cannot create a working binary.

Which version of ESPHome has the issue?

2023.10.0b2

What type of installation are you using?

Docker

Which version of Home Assistant has the issue?

2023.10.2

What platform are you using?

ESP32-IDF

Board

M5StickC Plus ESP32

Component causing the issue

i2s_audio speaker

Example YAML snippet

i2s_audio:
  i2s_lrclk_pin: GPIO0
  # i2s_bclk_pin: GPIO32

microphone:
  - platform: i2s_audio
    id: echo_microphone
    i2s_din_pin: GPIO34
    adc_type: external
    pdm: true

# speaker:
#   - platform: i2s_audio
#     id: echo_speaker
#     i2s_dout_pin: GPIO2
#     dac_type: external
#     mode: mono

voice_assistant:
  id: va
  microphone: echo_microphone
  # speaker: echo_speaker
  noise_suppression_level: 2
  auto_gain: 31dBFS
  volume_multiplier: 2.0
  vad_threshold: 3
  on_listening:
    - light.turn_on:
        id: led
  on_tts_start:
    - light.turn_on:
        id: led
  on_end:
    - delay: 100ms
    # - wait_until:
    #     not:
    #       speaker.is_playing:
    - script.execute: reset_led
  on_error:
    - light.turn_on:
        id: led
    - delay: 1s
    - script.execute: reset_led
    - script.wait: reset_led
    - lambda: |-
        if (code == "wake-provider-missing" || code == "wake-engine-missing") {
          id(use_wake_word).turn_off();
        }

Anything in the logs that might be useful for us?

INFO ESPHome 2023.10.0b2
INFO Reading configuration /config/m5stickc-plus-esp32-ac918c.yaml...
INFO Updating https://github.com/esphome/esphome.git@pull/5230/head
INFO Generating C++ source...
INFO Updating https://github.com/espressif/esp-adf@v2.5
INFO Updating submodules (components/esp-sr, components/esp-adf-libs) for https://github.com/espressif/esp-adf@v2.5
INFO Updating https://github.com/espressif/esp-dsp@v1.2.0
INFO Compiling app...
Processing m5stickc-plus-esp32-ac918c (board: m5stick-c; framework: espidf; platform: platformio/espressif32@5.4.0)
--------------------------------------------------------------------------------
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
 - framework-espidf @ 3.40405.230623 (4.4.5) 
 - tool-cmake @ 3.16.4 
 - tool-ninja @ 1.7.1 
 - toolchain-esp32ulp @ 2.35.0-20220830 
 - toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch5
Reading CMake configuration...
Dependency Graph
|-- noise-c @ 0.1.4
|-- Improv @ 1.2.3
Compiling .pioenvs/m5stickc-plus-esp32-ac918c/src/esphome/components/api/api_connection.o
Compiling .pioenvs/m5stickc-plus-esp32-ac918c/src/esphome/components/api/api_frame_helper.o
Compiling .pioenvs/m5stickc-plus-esp32-ac918c/src/esphome/components/api/api_pb2.o
Compiling .pioenvs/m5stickc-plus-esp32-ac918c/src/esphome/components/api/api_pb2_service.o
Compiling .pioenvs/m5stickc-plus-esp32-ac918c/src/esphome/components/api/api_server.o
Compiling .pioenvs/m5stickc-plus-esp32-ac918c/src/esphome/components/api/list_entities.o
Compiling .pioenvs/m5stickc-plus-esp32-ac918c/src/esphome/components/api/proto.o
Compiling .pioenvs/m5stickc-plus-esp32-ac918c/src/esphome/components/api/subscribe_state.o
Compiling .pioenvs/m5stickc-plus-esp32-ac918c/src/esphome/components/api/user_services.o
Compiling .pioenvs/m5stickc-plus-esp32-ac918c/src/esphome/components/binary_sensor/automation.o
Compiling .pioenvs/m5stickc-plus-esp32-ac918c/src/esphome/components/binary_sensor/binary_sensor.o
Compiling .pioenvs/m5stickc-plus-esp32-ac918c/src/esphome/components/binary_sensor/filter.o
Compiling .pioenvs/m5stickc-plus-esp32-ac918c/src/esphome/components/esp32/gpio.o
Compiling .pioenvs/m5stickc-plus-esp32-ac918c/src/esphome/components/esp32/preferences.o
Compiling .pioenvs/m5stickc-plus-esp32-ac918c/src/esphome/components/esp_adf/esp_adf.o
Compiling .pioenvs/m5stickc-plus-esp32-ac918c/src/esphome/components/gpio/binary_sensor/gpio_binary_sensor.o
Compiling .pioenvs/m5stickc-plus-esp32-ac918c/src/esphome/components/gpio/output/gpio_binary_output.o
Compiling .pioenvs/m5stickc-plus-esp32-ac918c/src/esphome/components/i2s_audio/i2s_audio.o
Compiling .pioenvs/m5stickc-plus-esp32-ac918c/src/esphome/components/i2s_audio/microphone/i2s_audio_microphone.o
*** [.pioenvs/m5stickc-plus-esp32-ac918c/src/esphome/components/i2s_audio/speaker/i2s_audio_speaker.o] Source `src/esphome/components/i2s_audio/speaker/i2s_audio_speaker.cpp' not found, needed by target `.pioenvs/m5stickc-plus-esp32-ac918c/src/esphome/components/i2s_audio/speaker/i2s_audio_speaker.o'.
========================== [FAILED] Took 8.78 seconds ==========================

Additional information

The configuration compiles if I remove the speaker option from voice_assistant but fails to compile if I comment out the speaker section.

esphome / issues