search

gdoor-org / gdoor

Wifi adapter and bus protocol documentation for the Gira TKS Door System
https://gdoor-org.github.io/
GNU General Public License v3.0
26 stars 6 forks source link

Audio Support #20

Open mrtnkhl opened 6 months ago

mrtnkhl commented 6 months ago

There is currently no audio support planned until reverse engineering and interfacing with the digital bus is fully working. Creating this Issue as a placeholder for possible future work and to inform others who might be looking for audio (and video) support.

Per @DaSchaef the audio (and video signals) are modulated onto the wire in an analog format. It is suggested but has not been verified through testing that connecting a headphone amplifier via a coupling capacitor to the bus would yield audio signals. Likewise, connecting a line-out to the bus could allow sending audio signals to the (door) station. It is important to understand that analogue audio (and video) signals are only present after the digital bus command has been sent to the respective station to transmit audio (or audio and video) on the wires that also transport the digital bus signals.

See https://gdoor-org.github.io/documentation/protocol.html "Request Audio", "Request Video" for more details on the respective bus messages.

Since is (unfortunately) no digital audio (or video) signal are present, a combination of an ADC and DAC, most likely connecting via I2S to the ESP32 is needed to convert the analog signals to the digital domain. Using an ADC, DAC and a Raspberry Pi could be an option as well but this would require too many resources and power.

Per @DaSchaef, on the ESP32 side, there is additional work needed, mostly in software, like working with the DMA engine, timing Wifi transmissions correctly, etc.. This work can be summarized in the categories of coupling, digitizing, processing and streaming to an endpoint, e.g. via SIP or RTSP. If one wanted to have bidirectional, parallel communication, with the other (door) station some analog circuit magic to prevent audio feedback loops would be needed as well.

mrtnkhl commented 6 months ago

https://github.com/sikorapatryk/sip-call

Some previous work has been done using an ESP32 with an ADC and DAC as a SIP client, so maybe this can be of use here.

mrtnkhl commented 6 months ago

https://github.com/vvigilante/comelit-simplebus1

Another similar project but for Comelit, including the analog circuits interfacing with the ESP32 internal ADC and DAC for audio:

image

mrtnkhl commented 6 months ago

See this thread for a similar discussion for a Comelit system: https://github.com/Elektroarzt/simplebus2-mqtt-bridge/issues/3

DaSchaef commented 5 months ago

I successfully received audio via my computer:

My voice was easily recorded on the computer. So I can confirm that bus audio is just plain simple analog "line audio".

I would not recommend this setup so far, as for a real usage more protection and also a bit of filtering is needed. In real world usage there would be multiple issues:

(both easily solvable)

After doing a lot of research:

My feeling is we should continue the audio and video route by just making plain simple adapters. Kind of "bus to cinch analog video" and "bus to cinch analog audio". These signals can be digitized with ~10€ USB Dongles and a Raspberry Pi, which itself can then act as VOIP phone, image sender or whatever is needed.

grafik

DaSchaef commented 5 months ago

I have no Gira equipment here to test video. (if someone finds a cheap used video outdoor station :wink: , indoor is not needed for tests, we know how to request it :sweat_smile: ) So my raspberry pi solution / hat would be a "blind try" regarding video.

mrtnkhl commented 5 months ago

@DaSchaef do any of the Comelit circuits posted above look useful? It seems like they have bus-coupled audio ADC/DAC working and code for it as well on the ESP32 platform?

mikburnz commented 1 month ago

door opener works perfect. instead of 2 way audio it would be helpful to send one predefined message to the speaker. So i can advice the delivery to store items in front of my door f.e. Any suggestions if we can realize that?

DaSchaef commented 1 month ago

This should be doable as well, I need to find a bit of time for it to make a ESP hardware. Currently I'm (very slowly) working on understanding ESP audio system. The most work is in the firmware/software, less in a hardware adaption. I want to first understand what is needed and also doable in firmware/software so that the hardware does not lack a functionality needed. (If we find someone with deep (!) ESP audio knowledge, better experience, it could speed this up. )

DaSchaef commented 1 month ago

I'm currently finishing a first try of hardware for audio support, yet I'm still struggling with firmware. The goal is that the GDoor adapter behaves like an ESP audio development board, so I can re-use the ESP Voip code etc.

SD Card (and therefore playback) is more tricky, as with the current ESP board I am a bit limited by I/O pins.

https://github.com/gdoor-org/gdoor/tree/audio

DaSchaef commented 1 month ago

Everyone who is interested and brave (it is very early alpha!), please tell me, I'll create a private repo to exchange infos about price/payment etc. Currently I'm still doing the PCB layout and final schematic checks.

mrtnkhl commented 1 month ago

I would be interested! ESPhome has audio support, e.g. to play out audio files. Maybe that would be an easier start?

jschroeter commented 1 month ago

I'm also interested!

mikburnz commented 1 month ago

I’m interested.

HJoentgen commented 1 month ago

Im also intereseted. I will ne installing the Gira system next week

DaSchaef commented 1 month ago

PCBs arrived today, now I need to test the firmware for audio support (which will also take a bit of time). I will keep you updated.

grafik

DaSchaef commented 1 month ago

I got a partly success, but it seems the ESP board is too weak (not enough memory) for the VOIP audio code. Next I'll try with a WROVER module instead of WROOM, it can be patched to the existing PCB. ESP documentation is really bad :-1:

https://www.amazon.de/dp/B09Z2GCCJZ

DaSchaef commented 1 month ago

With the new WROVER board I got a first voip connection (without being able to check audio so far). So at least the ESP code has no RAM memory issues anymore.

Testing VOIP via my FritzBox did not work, but it think the error is my FritzBox VOIP settings. Next step will be to setup a local debug VOIP Server so that I can start to test audio and debug also with server side logs.

mrtnkhl commented 1 month ago

@DaSchaef super cool! Can you go into further details on what your added in the v4.0 revision of the board? How is the setup looking on the HW and SW side of things?

DaSchaef commented 1 month ago

Currently I'm using above shown custom PCB + the above ESP (Amazon Link) connected via jumper cables (looks way more complicated than it is, it is a simple connection, no soldering or whatsoever).

I can now call my FritzBox from the ESP (GDoor Adapter). Audio is not working yet - this is now the next step.

grafik

Software is based on Esspressif ADF (https://github.com/espressif/esp-adf), I'm working in a private repo at the moment, as it is still very early/ugly, unstable and not working :) I'll merge it in the branch here once I'm sure that it is at least barely usable.

DaSchaef commented 1 month ago

For feature/HW changes in the v4.0 HW:

mrtnkhl commented 1 month ago

@DaSchaef very exciting! Thanks for the update! Looking forward to reading about the first successful 2-way audio call!

DaSchaef commented 3 weeks ago

I'm still struggling with audio, currently I only get silence in both directions - currently it looks like the S/W ist the reason and not the H/W. At least GDoor and VOIP seem to work in parallel. The ESP audio libs are a mess, but it is very slowly progressing - next step will be a small audio test script - not from the ESP libs, but from scratch - to have a minimal example to debug the missing audio. :sweat_smile:

DaSchaef commented 3 weeks ago

Still no audio but I have an assumption:

The audio codec needs a MCLK generated by the ESP32. The older ESP32 (incl. my boards here) can only output the precision MCLK on IO0, which I designed in HW. But it seems the ESP boards use IO0 themselves for their "Bootloader Button", therefore it can not be used although it is accessible on the pin header. :rofl:

From my research the newer generation ESP32S3 is able to freely route high precision clock outputs to any of the GPIO. So next try will be with yet another ESP board https://www.amazon.de/gp/product/B0CQNB74R9

DaSchaef commented 2 weeks ago

Good news, audio via VOIP is working.

I'll give more details about shipment next week. We need to try this in more installations before finalizing.

You will need:

  1. I will document the connection GDoor to ESP Board (via the jumper wires) in the next days.
  2. You need to solder a small bridge (very easy) so that the ESP board outputs the 5V from USB to the GDoor Adapter 4.0 grafik
DaSchaef commented 2 weeks ago

pinout

Connection Scheme

mikburnz commented 2 weeks ago

Awesome! Waiting for shipment instructions

stich86 commented 1 week ago

this is a good news do you think, after a reverse, if possible to include Urmet 2Voice BUS support to your product?

It's a 2-wire bus not polarized (running at 48/50V), and for sure video and audio are analog.

Thanks in advance for your amazing work!

DaSchaef commented 1 week ago

Hi all, I added the people interested to a private repo, I will create a issue there.

@stich86 Please make a separate issue and give a bit more details about the bus, Let's discuss there. My first impression would be try it with a USB audio interface first :)

DaSchaef commented 1 week ago

@mikburnz @HJoentgen @jschroeter Please check for the Github Invite, check the issue in this private repo and give feedback there or via mail. So that I can send out this week. :)

jschroeter commented 1 week ago

The Amazon link to the ESP board stopped working?

DaSchaef commented 1 week ago

https://www.amazon.de/dp/B0CYZVQR27