hoglet67 / RGBtoHDMI

Bare-metal Raspberry Pi project that provides pixel-perfect sampling of Retro Computer RGB/YUV video and conversion to HDMI
GNU General Public License v3.0
836 stars 112 forks source link

Proof Of Concept: Audio Signals into HDMI-Output (Muxing) #177

Open ZuiMelanieForno opened 3 years ago

ZuiMelanieForno commented 3 years ago

Hello.

I came here from the Amiga-Digital-Video @c0pperdragon, where I made a Featurerequest.

The Issue over there, was reopened and I was sent here to do further "investigation". ;)

What is the situation?

There is a PCB what @c0pperdragon provides, und @IanSB are doing there some kind of Magic on the "Chip´s", to intercept digitaly RGB into the RPi Zero.

What I saw is that there are unused Pins, that could possibly be connected to another Signalsources. Like Audio, in form of digital Signals or maybe (not) amplified Analog Signals.

On the Amiga Side there are two differen Chip´s to do the Job. The one for the RGB is the Denise and the Audio is on the Paula-Chip (Pins 30 and 31, digital).

@c0pperdragon mentioned that maybe the "thight timing" wouldn´t allow to add another (asychron/synced) Signal into the RPi Zero?

I have not the "expertise" in E-Engineering or "Amiga-Chip-Stuff" at all, but if I understand it correctly, there could be a possibility to "mux-in" more to the HDMI-Output then it seems...

So my Issue or rather a Question is:

Can be different types of signals mixed into the HDMI, by muxing Audio into the HDMI-Data-Frames, for instance?

Maybe it could be achieved by using I²C or some AD/DA-Conversion IC´s as (drivers) additional Hardware for preprocessing (and if needed syncing) with the timing on the RPi Zero.

The Idea came from @Crushless to use I²C, so I dunno nothing about "Computahs"... ;)

@hoglet Thanks in advance and also many more thx for this nice Project and wunderfull work.

c.u Zui from Hamburg, Germany

hoglet67 commented 3 years ago

@Hoglet Thanks in advance and also many more thx for this nice Project and wunderfull work.

FYI @IanSB is doing all the work now (and has been for at least a year),

I don't believe capturing audio on the Pi at the same time as capturing video is feaible. The memory timings are already incredibly tight, especially at higher bit depths.

There is also the issue of sample rate. As far as I understand, Paula supports variable sample rates (upto about 29KHz). HDMI audio needs to be at 48KHz. So a digital only solution would need sample rate conversion, which is not feasible in software (in the time we have available). It would also likely add significant lag.

(I have implemented a HDMI audio pipeline in the BBC Micro FPGA port for the Spectrum Next. Without proper sample rate conversion it sounds awful)

That leaves using an ADC to resample the analog audio, somehow feeding that into the Pi (I2S), and then encoding and muxing this into the HDMI stream. I just don't think there is the horse power to do this, without distrupting the video capture process. And it's unlikely to sound very good, as the ADC will be in a very noisy environment.

It's much easier just to take an analog audio cable from the Amiga to the monitor; most TVs support a seperate audio input. Or even better, use an external amplifier and speakers!

Unless @IanSB thinks otherwise, I think we should close this issue.

Dave

c0pperdragon commented 3 years ago

I did some research on the matter, and the Amiga generates the analog audio inside he Paula chip, never exposing digital waveform data. So an ACD would be the only option. I totally agree with Dave here that mixing audio into the HDMI is just not feasible with this special low-cost approach.

ZuiMelanieForno commented 3 years ago

Hello.

I ordered two Raspberry Pi Pico Microcontrollers.

They have 3 Pins for Analog Input. Maybe it could work out of the Box, but i assume there will be some extra Ciruit to make an Audio-Grabber. When (If) I can get a decent Input, I will try to couple that with the RPi Zero to see if I can get somehow a Network of I²c or such... :)

@JanBeta is willing to help, and if he can find the Time he will do a Oscilloscope Read of the Audio Pin´s.

So, stay tuned on 44.1 kHz for more Audio ;)

c.u. Zui

Breeffeehey commented 3 years ago

Hello,

Sorry if that is a stupid idea, but... what about using a cheap USB Audio SoundCard to capture the audio output of the Amiga and pass it through the HDMI, along with the video?

In my setup, computers and consoles are plugged via HDMI to my 5.1 Home Theater System... So, HDMI for Audio & Video is more simple...

What do you think?

I already have a USB Audio SoundCard ordered and on the way to test it, if someone is willing to help me figure it out...

c0pperdragon commented 3 years ago

I don't think using USB is possible here. First reason is that the program is running on the "bare metal", which means, there is no operating system. that could handle the USB port. So everything would have to be re-implemented. Second reason is that the single CPU core is already busy with reading the GPIO ports with little or no extra time to spare for additional tasks.

Breeffeehey commented 3 years ago

Thanks, I will look for an alternate solution...

Would be nice if there was HDMI+R/L --> HDMI converter, but I can see how stupid this idea is...

I've seen DVI or VGA + R/L --> HDMI converter, which I could use with a DVI to HDMI adapter, to take the HDMI Video only from the RPi and mix in the the audio... but have not found those under $50 or $90...

Thanks again for the answer

IanSB commented 3 years ago

@Breeffeehey

Would be nice if there was HDMI+R/L --> HDMI converter, but I can see how stupid this idea is...

It isn't a stupid idea and such things exist. Look for HDMI Audio embedder or HDMI Audio inserter.

There are also older products that convert DVI + audio to HDMI such as this: https://www.ebay.co.uk/itm/273936861003 I have one of those and it works OK (RGBtoHDMI outputs a DVI compatible signal by default)

Breeffeehey commented 3 years ago

Look for HDMI Audio embedder or HDMI Audio inserter.

I will!

There are also older products that convert DVI + audio to HDMI such as this

Yeah, I have a VGA + Audio to HDMI I use with a MacMini... those are cheap, but DVI + Audio to HDMI are $75 (CAD) and up...

I'm going to check when I get home... I have another VGA-HDMI with sound, but I don't remember which one-way it is (I don't remember if it's HDMI-->VGA or VGA-->HDMI...) Depending which way it goes, I'll check my options...

Thanks again for the suggestions!

Breeffeehey commented 3 years ago

Would there be a problem if I use the RGBtoHDMI Pi's USB port to power the 5V required by the VGA2HDMI in my A3000? Could this work, or be totally out of question and might lead to problems? (only problem I, in my somewhat limited knowledge of computers and electronics, can see would be some underpower of sorts in regions of the computer, including the Pi, but how much would it affect it?)

Thanks for your help, if it's not recommended, I'll have an external power source for the VGA2HDMI 5V requirement.

[reminder: I go from RGBtoHDMI-Pi's HDMI to an HDMI2VGA adapter, then through a gender changer to plug into a VGA2HDMI, with sound (guess that's where it needs the 5V USB), and this goes to my Amplifier, which distribute the sound to the speakers and the video, through HDMI, to my TV]

c0pperdragon commented 3 years ago

The USB-port of the Pi directly carries the 5V rail of the Amiga mainboard. So it could work, but it will draw power through the Denise socket, the Pi adapter and then the Pi itself.
The Pi itself should also work when the input voltage is a bit lower, because it regulates its own internal power down to 3.3V anyway.

Breeffeehey commented 3 years ago

I will try this weekend, then, and report my results!

Breeffeehey commented 3 years ago

Conclusion (sorry, it took little bit more time than anticipated...) : it's Ninjaneering, and it all works fine!

RGBtoHDMI in video slot of the A3000 miniHDMI out to HDMI --> HDMI to VGA adapter --> VGA gender changer --> VGA+Audio to HDMI Adapter --> HDMI out of the case

R+L Audio cable take the sound from the R+L ports on the back of the case to the headphone jack in the VGA+Audio to HDMI Adapter

Mini-USB of the Pi0 is powering the VGA+Audio to HDMI Adapter!

Pictures to prove it!

Now, only have to solve that occasional line of pixel glitches, like Yan Beta use to get in his videos... it comes and goes at random... will check for update on the software part and also check for the setting he used... DSC_0978 DSC_0979 DSC_0980

c0pperdragon commented 3 years ago

What an amazing hack! And it seems you pulled this off without doing any soldering at all! Just plugging together existing cables and adapters of all sorts.

Can you say anything about the video quality? Does converting the signal to analog VGA and back again cause any image noise or blurry edges?

Breeffeehey commented 3 years ago

Actually, I can't tell the difference between straight RGBtoHDMI in TV vs through this ninjaneering... Only thing is the occasional line of glitch, which is random (timing, but always at the same line... maybe I should investigate...)

Maybe it doesn't switch to analog lonv enough (it's just to go through a gender-changer... Or maybe it is because VGA can handle higher resolutions?

Maybe on 4K we would see a difference, but straight VGA from the A3000 connector to a monitor on the side, or through this, it's the same quality... i.e.: sharp as hell!!

And I have sound!!

c0pperdragon commented 3 years ago

When the visual glitches are caused by the Pi occasionally missing the sample points, this can sometimes be improved by slightly overclocking the Pi. You can do that directly from the settings menu.

By the way, I think the most amazing part of your "ninjaneering" is the abuse of the Pi's USB port to get power for the VGA-to-HDMI. Would never have thought of this possibility.

qbancoffee commented 2 years ago

@Breeffeehey , thank you for this work around, I confirmed that it works with my CoCo and the picture is still flawless. I bet the same idea but using a DVI to HDMI converter with audio in would be better since there is nothing done to the video signal. Anyhow I used an $8.00 VGA+audio to HDMI converter and a $12.00 HDMI to VGA converter and it worked. I put up a video of it in action if you want to see it. https://youtu.be/lIacGn5m7bw

Breeffeehey commented 2 years ago

@qbancoffee glad it works for you too! I've checked your video, and it's nice that your adapters don't need a 5V input to work! You want the funny part? I've undone my ninjaneering in my A3000... Since it can output directly in VGA, why go through the RGBtoHDMI, convert to VGA and back to HDMI? So, I plug in the TV directly via VGA... Or I could convert the VGA + sound to HDMI... I'm going to convert thi hack to my A1000 instead... eventually... with a way to combine the RGBtoHDMI and the PiStorm HDMIs...

Let's all keep having fun with our veteran computers!!

qbancoffee commented 2 years ago

@Breeffeehey actually the VGA to HDMI adapter needs 5v but I didn't talk about that because I was excited that it was working. So you undid your mod? Well at least it inspired me and hopefully others to try this. This is a great piece of ninjaneering and I think it's very creative. As you heard, I think my setup is rubegoldberghish so I would like to clean it up to look like what you have in your pics.

and yes let's keep having fun with our old computers!

Breeffeehey commented 2 years ago

@qbancoffee oh, I see... did not noticed on my phone... ;) Yeah, I undid my mod in the A3000... Since it already does natively output a VGA signal, I found it redundant to have a card capturing the video signal, convert it to HDMI through the Pi, convert this HDMI to VGA just to be able to grab this VGA and the Audio and then reconvert it all to HDMI with Audio... I'm going to grab the VGA coming from the computer directly, with the audio, and then convert it to HDMI... eventually... I will see in a future setup where I'll have to find my 5V... for the moment, my TV have a VGA+R/L input, so that's where the A3000 is plugged... Although, I do keep this mod in mind for the A1000... Hey, I had a good idea, better put it to good use... who knows when I'll have another one! ;) ;) ;)