search

jaku / SnapCameraPreservation

MIT License
577 stars 77 forks source link

[Idea] Package this as a plugin for OBS Studio #11

Closed nileshtrivedi closed 7 months ago

nileshtrivedi commented 1 year ago

This is not a feature request because what I'm suggesting is A LOT of work. But I think it would be worth building an open-source version of this app. OBS Studio already works as a virtual camera. If we can extract the ML models from Snap Camera app, it should be possible to support all the lenses as filters/effects in OBS.

Apologies if this discussion is not welcome here. Feel free to close.

MissCurlyBlonde commented 1 year ago

God I'd love for this to be possible.

jaku commented 1 year ago

Leaving this open just because this repo will most likely get the most eyes and attention for something like this.

This goes beyond the scope of this project and while it would be great, it would take a lot of work as you mentioned.

Anyone that is reading this and hoping for a solution, try to not get your hopes up as it would be a huge undertaking and would need some talented people to come together to complete it. I don't mean to be a pessimist on this, but it'd most likely be easier to rebuild something and get a new open lens format open than take on a project like this.

nileshtrivedi commented 1 year ago

a new open lens format

IMHO, it's the machine-learning models to track body parts which are the valuable bits in Snap Camera. It can be hard for a solo dev to build, train and optimize similar models from scratch.

YorVeX commented 1 year ago

a new open lens format

IMHO, it's the machine-learning models to track body parts which are the valuable bits in Snap Camera. It can be hard for a solo dev to build, train and optimize similar models from scratch.

While I never played with one myself I think there are quite a few open source solutions that can already do this, the most popular one being OpenCV. So maybe this part is not even as hard as we might think right now?

The other important part would be to be able to read and apply the lenses to that face/body model.

One option to achieve that would be to decipher the lens format and then code something to apply this. On the deciphering part it seems someone is already making good progress on this. The applying part might be a different story though.

Another option would be to write a third party tool or, as suggested here, an OBS plugin, that invokes Snap Camera's library files, something that also someone has started to look into.

So the options are there, but I still agree that we shouldn't be too optimistic about it for now, it most probably is still a long way to go.

And to be honest, while the actual lens technology is great, the software around it never was. No API but hotkey fiddling instead (I actually wrote myself a tool that can be controlled by HTTP requests and will then locally simulate hotkey presses to control lenses so that I have an HTTP API for Snap Cam that I can control from the outside) and the worst part is the ever changing delay which makes proper A/V syncing pretty much impossible. And sometimes the program would freeze when closing it and in the process write only half the settings file so that it's broken and you lose all your configured lenses (and hopefully have a backup).

So I don't know if just starting with something new with an open format wouldn't be the best option anyway.