Open Hedda opened 6 months ago
Hedda
commented
5 months ago Any feedback or input on these feature request ideas about Matter Casting support fod audio-only streaming and multi-room support for syncronized music stream playback in multiple rooms?
Hedda
commented
5 months ago FYI, chrisdecenzo wrote that there is now an official effort within the Matter working group right now to define use cases for audio players / smart speakers but only Matter/CSA members can participate, see comment -> https://github.com/project-chip/connectedhomeip/issues/31389
Hedda
commented
3 months ago FYI, looks like they updated the documentation for the Matter Casting APIs as part of tv-casting-app to match v1.3 Matter APIs:
https://github.com/project-chip/connectedhomeip/blob/master/examples/tv-casting-app/APIs.md
They also updated their tv-app (Matter TV Casting app example):
https://github.com/project-chip/connectedhomeip/tree/master/examples/tv-app
However these again so far only cover video casting and not audio-only for pure music playback.
Describe the problem you have/What new integration you would like
Please consider adding an native ESPHome component so that ESPHome-based smart-speakers and smart-displays in the future will be able to act as an A/V-receiver endpoint and player for "Matter Casting" (a.k.a MatterCast) audio/music and video streaming.
"Matter Casting" is a new open audio and video streaming protocol specification that is only a small part of the currently much-hyped Matter connectivity standard suite for IoT which is being developed by the CSA (Connectivity Standards Alliance) and its very impressive list of member companies.
This new audio and video streaming protocol specification is still in relatively early development and the work on it is being lead by Amazon, and Amazon is so far the only large company that publicly stated they will be supporting this new casting protocol (announced at CES 2024):
As such please see this as feature request that is requesting that ESPHome and Home Assistant developers looking get involved early in the developing of this new audio and video streaming protocol specification, even at the architecture level now that Nabu Casa is a member of the CSA, as I think that this is realative now more than even when you are looking into making voice-assistant hardware and software pipeline, as I believe this feature belong both there and in the ESPHome firmware, (at least when looking at achieving local-control but similar functionality compared to commercial smart-speakers and smart-displays like Google Nest and Amazon Echo products).
Please then consider developing a native "Matter Casting" receiver component that could in the future be used inside ESPHome so that an ESPHome-based smart-speaker or smart-display (with or without voice-assistant) can be used as "Matter Casting" audio and/or video receiver for picture and/or music streaming.
Currently, it looks like the "Matter Casting" specifications are still in early development and so far it seems to me like they only added video casting with "Video Player Architecture" with "Casting Video Player device type" and "Video Player endpoint" (with "Casting Video Player" and "Basic Video Player" functions). As such it still looks to be missing separate Matter architecture for an "Audio Player Architecture"? Therefore I submit this related feature requerst to the Matter (formerly Project CHIP) project on GitHub asking for them to also add an "Audio Player Architecture" and adding a new "Casting Audio Player device type" and "Audio Player endpoint" ("Casting Audio Player" and "Basic Audio Player"):
They lack example similar to
tv-casting-appbut perhaps instead for audio-only casting of music, sospeaker-casting-appor?Also missing example audio player app similar to
tv-appbut for music playback (preferably with multi-room synchronization)?I would like this feature request to also include audio synchronization between several different speaker systems for multi-room audio.,
Note! This feature request for "Matter Casting" should not be confused with feature request for IoT Matter Application Layer -> https://github.com/esphome/feature-requests/issues/1430
Please describe your use case for this integration and alternatives you've tried:
The alternative to using the Matter Casting open standard would be to create some other standard audio target for music stream using either DLNA or some reverse engineered proprietary streaming protocols like Apple AirPlay or Google Cast (Chromecast Casting), use use some middleware solution with conversion for indirect streaming via something like the Music Assistant 2.0 addon for Home Assistant by @marcelveldt
https://www.home-assistant.io/blog/2024/05/09/music-assistant-2/
I hope this is a good idea and that you will consider getting early into the development of this new open protocol that might become eventually a larger industry standard in the future as it is not proprietary like the competing Google Cast / Google Chromecast and Apple AirPlay.
Additional context
FYI, please see this as just an informal feature quest as an open discussion about this new open protocol that might become eventually a larger industry standard in the future as it is not proprietary like the competing Google Cast / Google Chromecast and Apple AirPlay.
"Matter Casting" is primarly intended to be a unified technology that allows you to cast video and/or audio content from your smartphone, tablet, and computer to your TV or smart display in a standard way that is suppose to support manufacture interoperability.
From an end-users point-of-view "Matter Casting" should will work the same thing as Google Cast and Apple AirPlay with one major difference: Matter Casting is a royaltee free and open protocol standward and will be based on common open-source code. That means it is a "free" casting standard designed from the ground up for cross-platform compatibility (with both Android, iOS devices, and various computer operating-systems being compatible from the start), allowing any developer to implement it into its products. Hopefully eventually ensuring wider-spread compatibility between devices of different manufacturers.
So the reason why I think this is interesting is that it is the first attempt by a large technology consortium (the Matter) to create also an open standard for local video and audio casting that is meant for broader adoption across different ecosystems for interoperability:
https://en.wikipedia.org/wiki/Matter_(standard)
https://csa-iot.org/newsroom/building-a-standard-that-really-matters/
https://csa-iot.org/all-solutions/matter/
Matter in general already has hundreds of member companies, but understand that not everyone has publicly commited to every part of that standard, as most have only announced support for the IoT parts that is controlling smart devices, however support for additonal device types is now being announced regularly
https://csa-iot.org/newsroom/matter-1-2-arrives-with-nine-new-device-types-improvements-across-the-board/
https://csa-iot.org/members/
Anyway, Matter Casting has so far 'only' been adopted by Amazon who announced at CES 2024 (Consumer Electronics Show, which is the largest tech event in the world). that they will first add support to newer Amazon Fire TV and Amazon Echo Show series of smart display devices, and even it sounds as if they might initially only implement video streaming?
More interestingly, Amazon has now stated that they will not add support for Google Cast or Apple AirPlay casting to their devices:
https://www.aboutamazon.com/news/devices/amazon-ces-2024-announcements
https://csa-iot.org/csa_product/amazon-casting-video-player/
"Amazon is a long-time supporter of using open technology standards to give customers more choice over the devices and services they use in their homes. That’s also why Amazon is a founding member and key contributor to the Matter Standard. At CES, Amazon announced Matter Casting, which enables customers to cast content to Fire TV and Echo Show 15 devices directly from supported streaming apps on iOS and Android. Customers can begin watching a movie or browse for their next favorite show from Prime Video on their phone, and cast it to their compatible Fire TV device or Echo Show 15. This is an industry-first demonstration of implementing Matter Casting.
We are excited to work with Amazon to bring Matter Casting to the STARZ app. Matter has been easy to integrate and will provide even more options for customers to enjoy our extensive collection of original series and popular films on their device of choice,” said Alison Hoffman, STARZ President of Domestic Networks.
Matter Casting is available now for customers around the world who want to cast Prime Video content from iOS and Android devices to Echo Show 15. It will be coming to compatible Fire TV devices in the coming months, including smart TVs from Panasonic with Fire TV built in. We’re working with Plex, Pluto TV, Sling TV, STARZ, and ZDF to add Matter Casting support later this year."
PS: The specifications documents can be downloaded from here (spec doc links sent by e-mail instantly after submitting the form).
Disclaimer: Again, I simply posted this for discussion since hoping for industry-wide adoption of an open standard for video and audio streaming in the future. I do not own any Amazon devices myself and do not plan on buying any (as personally already deeply invested in Google hardware, but if I ever migrate to a different ecosystem then decided it must be based on open standards).