search

rsta2 / circle

A C++ bare metal environment for Raspberry Pi with USB (32 and 64 bit)
https://circle-rpi.readthedocs.io
GNU General Public License v3.0
1.86k stars 245 forks source link

USB audio device support #222

Open mikeoliphant opened 3 years ago

mikeoliphant commented 3 years ago

Any thoughts on the possibility of USB audio interface support?

It would be great to have a bare-metal solution on Raspberry Pi that could use a USB-connected audio interface instead of integrated I2S.

My application of choice is guitar effects processing, and I would love to be able to get Linux out of the way and go bare-metal to get the best performance at the lowest latency.

rsta2 commented 3 years ago

Unfortunately this is currently not planned. The problem is, that the USB host controller drivers do not support isochronous USB transfers. I thought about dealing with this, when I have much time in the future, but this is far away.

mikeoliphant commented 3 years ago

Yes, it is a non-trivial problem to solve and there aren't a lot of public implementations floating about.

I did find this, though:

https://github.com/Themaister/libmaru

And, of course, the Linux kernel...

sebastienNEC commented 3 years ago

Hello Mike, I have actually worked on something similar, over I2S though. I am using circle for low latency I2S audio processing, and I have statically linked-in a lot of the usual suspects in audio fx (guitarix, rakarrack, etc). I am currently working on streaming low latency audio via WiFi to have a distributed audio fx system...

sebastienNEC commented 3 years ago

Maybe I should open another topic but @rsta2 : I have hacked CI2SSoundBaseDevice to provide I2S audio input, but this is really quick and dirty. Is I2S input on your list possibly ?

rsta2 commented 3 years ago

@mikeoliphant I think, this library is Linux based, not an USB driver itself. Yes, there is the dwc_otg Linux kernel driver, probably the only reference for this purpose. I analyzed it eight years ago to implement the current Circle driver, but I let the isochronous mode out, to reduce complexity.

@sebastienNEC I have I2S input on my TODO list. How did you extend the class interface to provide the received sound data? There was no sound input support so far in Circle. It would be great, if you could send me your modifications as a patch file per email. I would give you the credits for being the author, of course. A question is, how I can test the I2S input. Can I use a second RPi as I2S transmitter?

mikeoliphant commented 3 years ago

@sebastienNEC - your project sounds very interesting. What hardware are you using for audio in/out?

@rsta2 I understand better now. I was just thinking of the code that talks to the USB driver to implement audio, but you are saying that there is also work to be done to actually implement isochronous mode for the RPI USB controller.

sebastienNEC commented 3 years ago

@rsta2 I've sent you an email :)

@mikeoliphant: I've used the (not available anymore) fe-pi audio z v2, and a Pisound interface, which has instrument level inputs. I've sent you an email if you wish to discuss in more details :)

rsta2 commented 3 years ago

I2S input support has been implemented on the develop branch. There is a test program i2s-input in the new test/ subdirectory, which receives I2S data (24-bit, 2 channels) and sends it to the headphone jack (PWM device). The sample/34-sounddevices can be used to generate the I2S data on a second Raspberry Pi.

@sebastienNEC Thanks again for sending the I2S input patch! It helped me very much, to get it working. I will also use your WLAN AP patch, this will come later.

sebastienNEC commented 3 years ago

You're welcome, I'm glad I could help :)

probonopd commented 2 years ago

It looks like @BenZonneveld has implemented i2s over USB UAC2 audio: https://github.com//MiniDexed_Midi_Master/pull/2

I don't know more about it but perhaps it could be useful for someone and a candidate for being backported into Circle?

rsta2 commented 2 years ago

Thanks for info. I will have a look on it.

rsta2 commented 2 years ago

As I see it, the referenced project contains an I2S to USB audio converter, which runs on the Raspberry Pi Pico. It cannot be ported to Circle.

probonopd commented 2 years ago

Ah now I understand. Thanks for the explanation and sorry for the noise.

rsta2 commented 2 years ago

There is some initial support for USB Audio class streaming devices on branch usb-audio. Currently only USB Audio class 1.0 devices (int1-2-0) are supported and only in output direction. These devices normally are USB full speed devices.

Using these devices on the Raspberry Pi 1-3 and Zero requires the system option USE_USB_SOF_INTR defined (default setting) and should work even better with USE_USB_FIQ defined. With the latter option defined, the FIQ is used by the USB host controller driver and cannot be used for other purposes (e.g. serial device) at the same time.

The sample programs 29-miniorgan and 34-sounddevices are prepared to work with USB Audio class devices. You have to define USE_USB in the file miniorgan.h for the first sample program, or add the option sounddev=sndusb to the file cmdline.txt on the SD card for the second sample.

You may also have to add the option usbpowerdelay=1000 (1000ms) to the file cmdline.txt, because some USB audio devices requires some additional time to come up after powering on. The default setting of 510ms may be not enough there.

Feedback is welcome.

probonopd commented 2 years ago

USE_USB_FIQ defined. With the latter option defined, the FIQ is used by the USB host controller driver and cannot be used for other purposes (e.g. serial device) at the same time.

Just to clarify, does this mean that when it is defined, then RX/TX (e.g., for MIDI) cannot be used at the same time?

rsta2 commented 2 years ago

@probonopd There is some chance, that the serial device can be used with the IRQ instead, when the FIQ is used by the USB host controller driver (option USE_USB_FIQ defined). You have to change the parameter TRUE to FALSE here to use the IRQ. The question is, if the serial IRQ is handled quick enough at any time, so that MIDI events from the serial interface do not get lost. This can only be tested.

rsta2 commented 2 years ago

The feature branch usb-audio has been merged to develop and will be removed soon.

probonopd commented 2 years ago

Thanks for the heads-up. MiniDexed will change back to develop at that point.

nickverlinden commented 2 years ago

@rsta2 this is amazing news, thanks for all the hard work. What are the chances of input be supported in the future? I'm using i2s audio interfaces, which at first i liked because i could build them into my enclosures. But now i'm feeling more like it would be more beneficial to use usb devices because they come in so many varieties and usually have all the stuff on board that you need (pro level inputs, good dacs, ...)

rsta2 commented 2 years ago

@nickverlinden Thanks for the feedback! The available USB audio support on the develop branch is currently very limited. I tested it successfully with an USB adapter from UGREEN (vend8c-14), which uses an Unitek Y-247A chip, and with an C-Media PD 552 adapter (venc76-1607). Both adapters are USB full speed devices and support the USB audio class 1.0 spec. I suppose there are more devices, which will work. I also have a more sophisticated ASUS Xonar U7 device available for testing. This one does not work yet. It supports the USB audio class 2.0 spec.

There is still much work to do to support more devices and audio input, which is planned. This concerns the USB low-level driver (isochronous input), the USB audio class driver (for version 1.0 and 2.0) and the sound device upper layer in Circle, which presents the sound devices to applications. This can only be implemented step-by-step. I'm currently in a planning phase, what to include in the next Circle release 45.

nickverlinden commented 2 years ago

Thanks for the update, much appreciated!

rsta2 commented 1 year ago

There is support for USB audio streaming devices (input and output) for the Raspberry Pi 4, 400 and CM4 in Circle 45 now. Unfortunately it turned out, that it is very difficult to support this with the existing Circle driver for the DWHCI USB host controller. I got the necessary isochronous USB transfers working there in some way, but not in all configurations. Especially there was a big problem, when an USB MIDI controller was attached at the same time. That's why I decided to restrict the USB audio support to the RPi 4 at the moment. Not sure, if this will change in the future, but I will keep this open, in case the RPi 1-3 and Zero will be supported later.

DatanoiseTV commented 1 year ago

I am interested in USB Audio Hosting and Bridging. E.g. connecting 2 devices via USB and passing through multichannel audio. Would this be feasible?

rsta2 commented 1 year ago

@DatanoiseTV I never tested more than one USB audio streaming device, connected at the same time, but I think, there is a good chance, that it will work with two devices on the Raspberry Pi 4. But of course you have to test it.

DatanoiseTV commented 1 year ago

@DatanoiseTV I never tested more than one USB audio streaming device, connected at the same time, but I think, there is a good chance, that it will work with two devices on the Raspberry Pi 4. But of course you have to test it.

Do you have any example code or could provide one on how bridging 2 UAC devices would work out on circle?

rsta2 commented 1 year ago

I will provide an example soon.

rsta2 commented 1 year ago

@DatanoiseTV Attached there is an example program, which bridges the sound, received from one USB audio streaming device, to another device and vice versa. I tested it successfully between an ASUS Xonar U5 device and alternatively a C-Media PD552 or Unitek Y-247A device on a Raspberry Pi 4B.

It does not work without distortions between the two latter devices, because they are USB full-speed devices (FS) and the integrated USB 2.0 hub of the Raspberry Pi 4B seems to have problems, when two FS devices have to be feed with audio data at the same time. This hub has only one transaction translator for the split transactions. So one used USB audio streaming device must be USB high-speed in any case.

It's important to enable the system option REALTIME for this example, before build. You can do this by adding the following line to your Config.mk file:

DEFINE += -DREALTIME

soundbridge.zip