search

palmerr23 / EtherAudio

Teensy Audio EtherNet
19 stars 5 forks source link

readme

# Teensy Ethernet Audio Library for Teensy 4.1

This is ALPHA code and may change without notice

Version 2.0.1

The Ethernet Audio library provides connectivity between hosts on an Ethernet network for audio, text and structured data (service) functions. It assumes a T4.1 with a native Ethernet connection on a class C network. While 10 Mbps connections are supported, 100 Mbps is required for more than four channels of audio.

The library uses the VBAN UDP protocol (https://vb-audio.com/Services/support.htm#VBAN) to transmit messages over IP-based ethernet using QNEthernet (https://github.com/ssilverman/QNEthernet/).

Audio streams are compatible with VB-Audio products such as Voicemeeter (https://vb-audio.com/), Talkie and Receptor on Windows, Android and IOS. Only Teensy Audio compatible, 44.1kHz INT16 PCM, audio is supported.

This library is distributed under the "AGPL-3.0-or-later" license. Please contact the author if you wish to inquire about other license options.

Contents

  1. Introduction
  2. Ethernet management
  3. Audio Streams
  4. Non-audio (Service) streams
  5. Examples
  6. Bugs & Limitations
  7. To Do
  8. For developers
  9. Other VBAN Sub-protocols

Introduction

AudioControlEthernet provides all user-facing functions and must be explicitly instanced. The underlying Ethernet and UDP packet management is handled by AudioControlEtherTransport which is invisible at the user layer.

The library provides MDNS, so hosts can be referenced by hostname.local as well as by their IP addresses.

The underlying network must provide DHCP services.

AudioInputNet and AudioOutputNet handle audio traffic, and behave like other teensy Audio Library input and output objects.

AudioInputServiceNet and AudioOutputServiceNet can transport either text or structured data and have user-defined fields to aid in message triage.

Host identification is handled transparently, with hostname to IPAddress matching

Noe of the objects in this library can take update_responsibility.

Using this library

droppedFrames(bool reset) provides the number of VBAN frames that failed to be processed since the last reset.

When an incoming queue grows longer than MAX_AUDIO_QUEUE, frames are dropped.

Similarly for outputs, for instance when there is a network disconnection. There does not need to be an active receiver for output packet streams.

Sample Code

// Connect to Ethernet but do no audio processing.
#include "control_ethernet.h"
AudioControlEtherNet   ether1;
setup()
{
  ether1.setHostName("Teensy1");
  ether1.begin();
  IPAddress myIP = ether1. getMyIP();
  Serial.println(myIP);
}
loop()
{
  // no code is required in the loop
}

Audio Streams

AudioInputNet and AudioOutputNet carry audio in 44.1kHz INT16 PCM (Teensy Audio) format.

They may be instanced with 1 to 8 audio channels. The default is 2 channels.

Creating input objects with more channels than required has little impact on memory or processing.

Output objects with more than five channels produce two VBAN packets per Teensy Audio buffer and consume twice as much queue memory.

Multiple instances of input and output objects are allowed, however each should have a distinct streamName.

Be aware of the CPU, AudioMemory and Ethernet bandwidth impacts of large numbers of channels as each channel creates buffers, a packet queue and a separate packet stream.

begin() must be called for each input or output instance.

As with other Teensy Audio objects, individual inputs and outputs may be left unconnected (no AudioConnections).

Subscribing to Inputs and Outputs

Only one subscription is permitted per VBAN stream for both inputs and outputs. Each stream must have a unique streamName. The same streamName may be used for both an input and output stream.

Only one AudioConnection is allowed to any individual output channel.

subscribe(streamName, hostName) connects to an incoming stream. hostName may be omitted for output streams where the stream is to be broadcast, or for inputs where there is only one other host emitting that streamName on the network. Currently this form of subscribe() is not enabled for output streams. Subscriptions by fully-qualified hostName are not yet supported.

Subscribe(streamName, IPAddress) acts similarly for output. For broadcast output streams, omit the second argument. IPAddress can either be a broadcast address, obtained with getBroadcastIP() or the full IPV4 address.

A stream only becomes active once it is subscribed. No outgoing packets are sent, and all incoming received packets are dumped, for inactive streams.

Subscriptions can be made prior to VBAN packets appearing, as there is a regular housekeeping function that tries to match orphan streams to subscriptions.

unsubscribe() frees the input or output stream.

Sample Code

// Connect to Ethernet and process audio packets
#include "control_ethernet.h"
#include "input_net.h"
#include "output_net.h"

  AudioControlEtherNet   ether1;
  AudioInputNet             in1(8);   // 8-channel input
  AudioOutputNet         out1(2); // 2-channel output
  // patchCords as required
setup()
{
…
AudioMemory(10);        // need at least the total number of active channels (AudioConnections)
  ether1.setHostName(“myHost”);
  ether1.begin();

  in1.begin();    in1.subscribe(“Stream1”);

  out1.subscribe(“myStream”);
  out1.begin();
…
}
loop()
{
  // no additional code is required in loop()
}

Non-audio (Service) streams

The Serve sub-protocol can send and receive structured or unstructured (text) data.

Structured data

Possible uses include the regular communication of a set of control parameters or audio levels for remote display.

Unstructured data

Text can be transferred in the same way as structured data. VBAN chat is explicitly supported using the VBAN_SERVICE_CHAT serviceType (see hdr.format_nbc).

Long messages

Multi-frame reconstruction is not directly implemented, so messages of more than 1436 are not supported. For instance, long MIDI SYSEX messages may be broken by this limitation.

Not yet implemented: Long messages may, however, be handled by user code. When writing extended packets, the optional argument extend may be set to true, and is returned as a header flag in packets received. The last packet in the sequence should set the extend argument to false.

Sample Code

// Chat with another host
#include "control_ethernet.h"
#include "inputService_net.h"
#include "outputService_net.h"
  AudioControlEtherNet   ether1;
  AudioInputServiceNet      inChat;
  AudioOutputServiceNet     outChat;

setup()
{
…
// AudioMemory is not consumed by non-audio traffic
  ether1.setHostName(“myHost”);
  ether1.begin();
  inChat.subscribe(“myChatStream”, VBAN_SERVICE_CHAT, myChatHost);   
  inChat.begin();
  IPAddress chatHostIP = {192,168,10,3};
  outChat.subscribe(“myChatStream”, VBAN_SERVICE_CHAT, chatHostIP);
  outChat.begin();
…
}
loop()
{
// Process incoming and outgoing chat messages – see VoicemeeterChat example.
}

Examples

MultiStreamAudio

This example requires two Teensy 4.1s or a single Teensy 4.1 and Voicemeeter.

A Teensy Audio Shield one Teensy is useful to monitor/display audio.

AudioControlEtherNet creates a static global AudioControlEtherTransport instance etherTran which handles network tasks and ethernet-facing queue management. This allows etherTran to be accessed transparently from input and output objects. Its static updateNet() function is hooked into EventResponder which runs at every call to yield(), at the end of each loop() and while delay() is operating.

Each type of VBAN input packet (audio, service and MIDI == serial) shares the same queue packet format.

samplesUsed is used for managing different VBAN packet and Audio buffer sizes for audio inputs and outputs. For incoming service packets it contains the length of the data payload.

Subscriptions

Subscriptions tie an input object to a host/stream of the same VBAN sub-protocol. Subscriptions may be made before an incoming stream becomes active.

If a new stream comes from a previously unknown host, a PING0 packet is sent with this host’s credentials.

If a PING ‘REPLY’ is received, matching inactive subscriptions are made active (updateActiveStreams()). This matching is also performed regularly by housekeeping called from updateNet().

Outgoing streams do not have subscriptions and have only a streamsOut entry and a queue. This precludes subscribe() by hostname for outgoing streams, which may be addressed in a later release.

Queues

MIDI messages may be transferred using the Service sub-protocol (untested).

Thanks

Special thanks go to Shawn Silverman for assistance in ironing out the network layer bugs.

Thanks also to Vincent Burel for the valuable assistance with all things VBAN.

Processed with https://products.aspose.app/words/conversion/word-to-md