search

Tympan / Tympan_Library

Arduino/Teensy Library for Tympan Open Source Hearing Aid
MIT License
116 stars 31 forks source link

USB audio not supported #34

Open hriday-mehta opened 3 years ago

hriday-mehta commented 3 years ago

Currently USB audio in/out is not supported by tympan. Any idea on when will it be supported?

chipaudette commented 3 years ago

Hi!

USB audio could be supported as long as you always use a 44kHz sample rate with 128 points per audio block. This inflexibility is due to the underlying Teensy code for USB-based audio. Re-engineering the USB code is beyond the scope of what we can do. But, if you can live with 44 kHz / 128 points, it can be made to work reasonably easily.

Would 44 kHz / 128 points meet your needs?

Chip

chipaudette commented 3 years ago

Here is an example sending audio from the Tympan via USB to the PC: FreqDomainDemo_liveUSB_BLE. This the sketch that I used for this video demonstrating voice-shifting / USB audio.

Note that it uses the USB audio blocks from the Teensy library and then uses Tympan blocks to convert back and forth between the Teensy audio formant and the Tympan audio formant ("_F32").

As of the current date (June 10, 2021), this example sketch requires that you use the "develop" branch of the Tympan library. And, again as of the current date (June 10, 2021), the develop branch requires that you update your Teensyduino to a beta version, which you get from the Teensy forum here.

chipaudette commented 3 years ago

I would like to wrap the Teensy's USB audio class in a Tympan "_F32" wrapper. I did this before, but it would cause compiler warnings if you didn't set your Arduino IDEA to have the "USB Type" include audio. Since most people didn't use the USB audio, this was an annoying warning, so I pulled the _F32 wrapped USB audio class out of the Tympan Library

I think that there is a way for our code to listen compiler flags to avoid these warning messages so that we know if USB Audio has been set in the Arduino IDE. If I can find out how to do that, we can put the _F32 USB audio class back in our library.

It's on the To-do list, for sure.

nithinr07 commented 3 years ago

Hi,

I am trying to add an AudioOutputUSB block to the single-channel WDRC code. However, this code can't really run well on the Rev D with the 44.1kHz and 128 block size setting, so I was wondering if there is any way to send the audio through USB at lower sampling rates.

Thanks, Nithin

chipaudette commented 3 years ago

The USB audio functionality is the exactly the same code as the Teensy Audio Library USB audio code. That code is limited to 44.1kHz and 128-sampke block size. Sorry.

Does the FreqDomainDemo example that I linked above work ok? Is your problem occur only when you use the USB audio with the WDRC?

nithinr07 commented 3 years ago

Yes, as long as the sampling rate is 44.1KHz and the audio block size is 128, it works, but unfortunately, the 8 band WDRC code can't support this on the Rev D I believe.

chipaudette commented 3 years ago

You are correct. the 8-band WDRC consumes too much CPU on the RevD to be run at 44 kHz. If you want the USB audio, you will have to use fewer bands.

It is possible that you could run the Tympan at 22.050 kHz / 64-sample blocks but you tell the USB audio code that you're running at 44.1k / 128. You would then have to write some custom code that you would insert just before your audio reaches the USB Audio class...your custom code would upsample your 22k/64 audio blocks to the 44k/128 rate required by the USB Audio. It seems to me like this should be possible, but I have never tried it. Therefore, there is no example code for you to follow. It would be a very impressive hack, though!

Assuming that you do not want to attempt such a hack, there is a group in Montreal that is looking to dig deep into the Teensy USB Audio code (and, therefore, the Tympan USB Audio code). They are trying to extend the code to enable the Tympan to send USB Audio at different sample rates. They might even be considering allowing it to send more than just two channels of audio. That would be quite an enhancement! If they do it, I believe they are planning on working on it in July - September.

nithinr07 commented 3 years ago

Also, just to point out, I noticed that there were distinct "clicks" in the USB output when the sampling rate was 44100 Hz. However, this problem was solved when I changed the sampling rate to 44117 Hz.

chipaudette commented 3 years ago

Yes, the requirement for 44117 is part of the limitations (and odd behaviors) that come with the Teensy audio library.

Sorry.

nithinr07 commented 2 years ago

Hello again!

So, I managed to make the Tympan run at 24,000 Hz / 128 sample block size by tweaking the Teensy library (the USB descriptors), but I wasn't able to lower the block size. I noticed that the WDRC code performs best at lower block sizes, so I was wondering if you could provide me with any leads regarding this.

Nithin

chipaudette commented 2 years ago

Wow! Congratulations on gettng the lower sample rate to work over USB! Very impressive!

As for shorter block sizes, the WDRC is actually more computationally efficient (ie, less CPU consumed) for the larger block sizes like 128. The only reason to go to shorter block sizes is if you care about latency.

Latency is the the delay that occurs between sound hitting the microphone and processed sound coming out of the earphone (see this blog post of mine). If you care to have a short latency, you would want to reduce the block size.

Do you care about latency?

C:\Program Files (x86)\Arduino\hardware\teensy\avr\cores\teensy3

Change AUDIO_BLOCK_SIZE from 128 to something smaller. Change your Tympan code to match this new value. Try to compile and run. Perhaps that will give you shorter blocks over USB?

Again, this is only relevant if you want shorter latency. Since your goal is to send audio via USB to a computer (or recording?), I might suggest to you that latency is not relevant and that staying with 128 points is maybe satisfactory?

Chip