search

hd-zero / hdzero-goggle

MIT License
263 stars 76 forks source link

[feature request]AI corrected signal loss. #311

Closed Shaun-York closed 5 months ago

Shaun-York commented 1 year ago

Would it be possible to train a computer vision model with OpenCV to fill in lost image data in realtime?

nurikk commented 1 year ago

And AI 4k upscale 😍

robhaswell commented 1 year ago

You would need to add a GPU into the video path. I'm certain this would be a fun research project but there is no chance of seeing this come included with HDZero.

lebed2045 commented 1 year ago

jokes aside, it seems possible to pre-train model to predict the most statistically optimal pixel color based on neighbours so actual pixel filaments can be done on fly with very low computation overhead. Another option, but I'm sure developers explored it already, is to dedicate small part of bandwidth to send colors of big areas first, i.e. like old JPEGs use progressive loading in the past.

xtr0d3m0n commented 1 year ago

Depending on how much hardware u got on v536. It got an NPU that can probably do some decent inference.

francoisbeaussier commented 1 year ago

Would averaging the neighbouring pixels already be an improvement over the current "multicolour static" behaviour?

Edit: found https://github.com/hd-zero/hdzero-goggle/issues/40

ligenxxxx commented 5 months ago

Any software-based video processing will increase latency.