[Feature Request] Ability to work with capture card sources and track mouse? (on a different machine)

[JoqniX]

hewwo! i've been using the the plugin! its satisfied alot! thank you for making it! it is really helping alot! i was wondering if it would be somehow possible to get the mouse coordinates from a different machine and make it work on capture card sources ? since i mainly use my game pc for my work streams and such, which needs this plugin alot. since there's alot of limitations if you're a dual pc user right? either i have obs running on game pc and send the feed via ndi or the obs preview window method etc are my only choice atm. ( since i'm trying to balance the resources load on game pc i usually not use obs ) but yea if it was somehow possible to do this without having obs running on game pc and make it send the mouse coords etc to stream pc's obs where the obs-zoom-to-mouse plugin reads it that would be really help alot!

since there isn't much solutions to this use case! but i think its definately might be possible somehow? right? do give me your thoughts too! and if there's others so find this and if you're an dual pc user like me do share your thoughts on this as well!

the following is what i think would work

so i was thinking like! some script or something which a user has to run on their another system ( for example - game pc) which tracks the mouse coords? and sends that info back to the stream pc's obs (where the plugin/script is installed!)

or maybe what if the user just installs the script like normally? and let it act as a remote thingy? that would work but taht would require the user on game pc to have obs running? ig for those who dont wanna run obs? can have a external application/script which they needa launch or something?

then a option in the zoom-to-follow script to connect to that script or some sort ( maybe you can do like a connect to a ip kinda? since most likely these machines would be connected over the locally via ethernet so ip might work ig? )

so that would mean if remote mouse follow is enabled? then original (current machine's mouse follow will be disabled?) maybe having an option for that would help alot ig!

but yea again do lmk if this could be possible! ofc since its a lua script, there would be limitations and stuff. but yea even if its not possible. if you have any ideas or know any other methods do lmk!

again thank you for reading this! hope you have a good day ahead! ✨

[BlankSourceCode]

OBS lua doesn't have much support in the way of getting external data. So it would be tricky to request/receive the cursor position from a remote machine.

Off the top of my head, you might be able to do something with a udp port in the script. You'd also need something running on the remote machine that sent the mouse position to that udp port, such as a C# application or similar. I have no idea if that would work, and if it did, I'm not sure how laggy such a communication would be.

It seems like a bit much to support out of the box, but maybe someone has a better idea on how it could work?

[BlankSourceCode]

@JoqniX I don't use a 2pc setup so I have a couple of questions on how this might work.

First this is what I think you are talking about:

• You have 2 machines:
  • OBS PC - machine with OBS installed
  • Game PC - machine without OBS installed
• You connect the Game PC to the OBS PC by plugging in an hdmi cable from Game PC into a capture card on OBS PC
• You add a new video capture source in OBS and point it at the capture card

Now your OBS is showing/recording content from the Game PC, and you want to zoom into something on the Game PC.

• You move the mouse on the Game PC to where you want to zoom
• You press a hotkey on the OBS PC(?) to toggle zoom

OBS zooms the video capture source into the position of the cursor on the Game PC.

I am assuming you normally control OBS stuff by interacting with the OBS PC and not the Game PC, so you would press the hotkey on the OBS PC to toggle the zoom. Is that true?

If you didn't mind also moving the mouse on the OBS PC, you can sort of get it to work with the current script today.

Here is how it could "work" today (if you don't mind using the mouse on the OBS PC):

• Setup your 2 machines as above, and add the video capture source on OBS
• In OBS, enable Allow any zoom source so that the video capture source is available in the dropdown
• In the Zoom Source dropdown, select your video capture source
• Enable Set manual source position
• Set the position and size settings to match the monitor that you are capturing on the Game PC
  • Let's say your Game PC is a single monitor using 1920x1080 which is also the same as your recording canvas size in OBS (if not you'd have to modify the settings below)
  • X = 0
  • Y = 0
  • Width = 1920
  • Height = 1080
  • ScaleX = 1
  • ScaleY = 1

Now when you press the hotkey, obs-zoom-to-mouse will zoom into the Game PC video capture source, but using the mouse coordinates from OBS PC. So if you move your mouse to 0,0 on the OBS PC, it will zoom into the top-left corner of the video capture source.

"But I can't tell where my zoom is going to go because I'm moving the mouse on the wrong PC" - Yes, this is true, so lets make it a bit easier:

• Add a new Scene to OBS
• In the new scene, add a new video capture (not existing) and point it to the capture card
• Right-click the new Scene in OBS and select Windowed Projector (Scene)
• Move and resize this window so that it sits at the same coordinates as the ones you set manually above

Now you can move the mouse on the OBS PC over the windowed projector and toggle the zoom hotkey. OBS will zoom into the original video capture using the coordinates on the mouse on the OBS PC, but because you have the game displayed in a window at that location you'll know where you are zooming into.

Is it a mess? yes. Does it probably require a lot of processing power on the OBS PC? also yes. Does it solve your scenario? eh.. maybe?

If you really want to move the mouse on the Game PC, here are my questions:

1. Did I get the setup in the first part correct? (Is that how your setup works with the 2 PCs?)
2. Do you need to be able to press the toggle zoom hotkey on the Game PC? (or is pressing the zoom hotkey on the OBS PC fine?)
3. Do you need this to work cross platform? (aka are you using only Windows machines, or is your game/obs machine actually running Mac and/or Linux?)
4. Is this a common way to record/stream (aka using 2 machines) and also wanting to zoom?

Sorry for all the text. If anyone else out there is interesting in such a feature, reply back and let me know.

[JoqniX]

hewwo! thank you for writing back! @BlankSourceCode

about your questions yu needed answers for!

1. I am assuming you normally control OBS stuff by interacting with the OBS PC and not the Game PC, so you would press the hotkey on the OBS PC to toggle the zoom. Is that true?

yea! i use the stream pc! mainly for controling obs stuff and fixing etc. and just keep the game pc for only to showcase what i do etc. such as playing a game, working on something, etc.

2. Did I get the setup in the first part correct? (Is that how your setup works with the 2 PCs?)

yea obs is installed on both pcs! i just use game pc's obs! to balance the loads as my stream obs always hits 100%cpu usage sometimes. since i started mulitstreaming its been trying its best to live xD!

but yea i usually do ndi feed from game obs on a green bg so i can put extra sources etc to balance. ( for example if im in need to show any infomative stuff i rather than using capture card ( which shows the whole screen) i sometimes use windowcapture etc on game obs and send it over via ndi to stream obs and have it there on stream! ( but usally i tend to not use obs on game pc as again resources stuff ;w; - since i gotta keep other stream apps open ( like vtube studio etc which does take alot of resources - i cant keep much apps open on stream pc as its already maxed out. )

3. Do you need to be able to press the toggle zoom hotkey on the Game PC? (or is pressing the zoom hotkey on the OBS PC fine?)

hmm i think both are fine! cuz like if its possible i could like link the hotkey to like sammi or streamerbot so i could use trigger via deck or chatcommand or heck even via midi xD etc.

4. Do you need this to work cross platform? (aka are you using only Windows machines, or is your game/obs machine actually running Mac and/or Linux?)

yea im on windows 11 ( 23H2 Build OS - 22631.2715 ) ( same for both! )

5. Is this a common way to record/stream (aka using 2 machines) and also wanting to zoom?

hmm tbh idk actually! like the main limitation is for those who use capture card is since it works like a screen (shows desktop etc) so there's hardly any control right!

but also another alternative to this is using ndi, teleport, obs beam etc etc. but for those who use alot of network stuff such as vban to route audio b/w pcs etc , and using ndi etc. and then streaming and also playing an online game ... it would be sooo pein cuz internet will be struggling right! for me i used to do that! but i had hit that limit. where my network couldnt handle at all, issues like using ndi on game pc = internet gets maxed out and cuts off completely on game pc moment, so i had to switch to hardware method = capture card!

then the issue at hand occurred! how would you zoom in and stuff! if you're on 2 pc setup? but also like i believe. for my usecase! its like i use my stream obs to have basic stuff such as mic, scenes , sources, etc route my ndi sources from game pc to stream obs. and having all browser sources etc etc. so i can balance out the system resources.

ig if it was someone else. they would do the usual method like - for example if they are making a tutorial etc. then they would use the pc they work on. ( ex - game pc) and record using obs in game pc and have zoom follow etc.

so i noticed someone else who kinda does the same method as i do!. i remember asking nutty bout this! months back! since he does use dual pc setup! but also his setup is way different too. i believe his response to my question was like " there is not possible solution for that yet afaik" so ever since i kinda gave up and stuff! until now i thought! maybe it would be possible somehow?

6. You connect the Game PC to the OBS PC by plugging in an hdmi cable from Game PC into a capture card on OBS PC & You add a new video capture source in OBS and point it at the capture card

yes that's what i do! as its a internal capture card. so its in the stream pc. and i connect via hdmi from game pc to capture card on stream pc.. so in game pc - so capture's card display ( monitor 2 ) would be mirroring my main monitor! ( since i only have 1 monitor for game pc , so its mirroring that! )

---

also regarding the method you mentioned above! yea i tried it! yea that would kinda work but idk if i could use that sadly ;w; cuz again resources issue ;w;! and on top of that im multistreaming to yt & twitch so extra resources getting used.. but also i can see if i can use this whenever possible!

but also it would be a bit hard to switch around with mouses! when you're actively doing a work and using game pc mouse then yu gotta remind yourself "oh i gotta move stream mouse too so others can see properly what im working."

---

then about the

udp port method! yu mentioned above! yea i could try that! ig ( im not much of a coder but yea ill try my besh to figure out how that works! ) but also like nowadays apps like sammi, streamerbot etc can like track mouse pos stuff right? i was thinking hmm! would it be possible to send data from let's say - udp port method , stream apps or even autohotkey etc to the the script somehow? or something?

but again sorry for the long texts. but also thank you for taking your time to read and understand! feel free to ask again if there was anypart yu kinda got lost or confused! and mainly thanks for helping out! <3 do have a good weekend ahead 💖

[JoqniX]

ill try to share this issue around so others could find it and also suggest us some good methods to tackle this! i hope its okie >///<

[BlankSourceCode]

@JoqniX It seems like you use your dual machine setup in many different ways. Sometimes you have OBS open on both machines, sometimes you don't, sometimes you use a capture card, sometimes you use NDI, sometimes you use a bunch of different software I've never heard of, sometimes you multi-stream, the list goes on.

I'm only one person, so I doubt I have the time to make something that works in all those different scenarios.

How about this, I tell you what I can do and you tell me if it would be useful or not?

Here is what I can make:

1. I update obs-zoom-to-mouse.lua so that it has a UDP socket server you can enable
2. I make a small application that sends the current mouse position to that socket server

This is how it would work:

1. Add the modified obs-zoom-to-mouse.lua on the OBS PC
2. Enable the socket server option
3. Add the small application to the Game PC
4. Run the small application and specify the OBS PC's ip address where it can send mouse positions to
5. Everything else is as normal:
   1. You select a zoom source in obs-zoom-to-mouse.lua on the OBS PC (I assume the capture or NDI source that is coming from the Game PC)
   2. You enable the Set manual source position option in obs-zoom-to-mouse.lua and set the X/Y/Width/Height to the position of the source on the other machine (since the script will have no way of knowing that)
   3. You press the toggle zoom hotkey on the OBS PC
   4. It will zoom into the selected zoom source
   5. Instead of using the local mouse position like it does today, it uses the mouse position from the Game PC
   6. As you move the mouse on your Game PC, the zoom on the OBS PC moves around

Would that be useful?

[JoqniX]

It seems like you use your dual machine setup in many different ways. Sometimes you have OBS open on both machines, sometimes you don't, sometimes you use a capture card, sometimes you use NDI, sometimes you use a bunch of different software I've never heard of, sometimes you multi-stream, the list goes on.

haha, yea i get asked alot. why do i complicate stuff alot for myself xD but yea it is what it is.

also regarding the ones you pointed out. i feel like those would really work~ and is useful!

once you're done with making the small application and modified zoomtomouse lua. ill test it out for you and provide you feedback on it okie!

and i feel like if ppl wanna use the normal zoom to mouse they can just duplicate/add another lua and use normally. while the first lua does work only for game pc.

or ig after afew tests and stuff. in the future ig a tickable option for like this feature would be good?

• [ ] Enable Remote Machine Mouse Tracking ( idk a better name ) and below it. you could maybe provide the link to download the small application ig. and ig a tool tip or info would work 🤔

so they wont needa duplicate the lua and have it all in one thing? ( but in a way if one wants multiple sources to be zoomable. they gotta do it right? so yea

but yesh again! thank you so so sooooo much! for working on this. and taking the time out of your day to focus on this. it really means alot. but do remember do take rests and stuff. and thank you again. do have a good weekend ahead~

[BlankSourceCode]

@JoqniX - Here is a new repo that contains the small program that you need to run on the second PC: obs-zoom-to-mouse-remote

You'll also need the obs-zoom-to-mouse.lua script from the remote branch here: obs-zoom-to-mouse.lua

Follow the instructions in the first repo, to make sure you put the ljsocket.lua file next to the OBS script, you should then see a new option to enable the socket server in the script. Running the other program and passing in the hostname will forward the mouse position to the script.

Good luck!

[JoqniX]

okie thank you! imma try testing it out now! thanks again for working on this! it really means alot <3

[JoqniX]

it works so well! im so thankful! thank you <3

[JoqniX]

ah i accidently opened it. il send the small bug i noticed to the issue i referenced before sorry bout this!