duhow / xiaoai-patch

Patching for XiaoAi Speakers, add custom binaries and open source software. Tested on LX06, LX01, LX05, L09A
GNU General Public License v3.0
186 stars 30 forks source link

Some questions #78

Closed nekromant closed 3 months ago

nekromant commented 3 months ago

Okay, not really an issue, but just a few questions about this project's internal design and goals, since I think I'm going to dive deeper and perhaps fix a thing or two and don't want to open any more issues.

  1. Was there any real reason to reinvent the buildsystem from scratch instead of using buildroot/yocto/openwrt that can build rootfs from scratch?
  2. Right now rootfs is in squashfs and changing things like adjusting hostname, enable/disable services and etc. can only be done using either rebuilding or via maintaining symlinks to /data. There seems to be overlayfs in kernel though. Why not mount /data on top of / for read/write access? (pretty much the way OpenWRT does usually).
  3. The current Assistant implementation although works pretty good, is practically useless if the speaker itself is playing audio. Are there any plans/ideas on how to implement echo cancellation based on stream being played by the speaker?
  4. And the next big problem is beamforming. Was there any research on that? Without that in a noisy room it would be very hard to pick up speach. According to /proc/asound:
    access: MMAP_INTERLEAVED
    format: S32_LE
    subformat: STD
    channels: 8
    rate: 48000 (48000/1)
    period_size: 2048
    buffer_size: 4096

    We have up to 8 microphones (6 seem to be actually wired), so theoretically ODAS should be possible at least on LX06. Although, we still have to figure out microphone configs. I have some crude scripts from RPI Respeaker HAT that can be used, but I'm no big DSP expert ;(

duhow commented 3 months ago