Adding audio to stream

[botiapa]

I found a fork that introduces audio into the stream. Maybe this could be implemented in this as well. Here's the fork: https://github.com/hikkamorii/hikkaAVStream

[ShayBox]

I'll look into this, that would be useful to have

[botiapa]

Cool, we do have to keep in mind that this would introduce some additional dependencies tho.

[ShayBox]

If you have discord you have pulseaudio

[ShayBox]

It seems like that fork just has some copied commands from a linked repo that aren't actually configured to send output into input, but input into output, so I'll have to do more messing around

[botiapa]

I think this shouldn't be enabled by default. We should add this as an argument.

[ShayBox]

Yeah I was thinking the same, if it needs a dependency and isn't required for the core functionality I want it to be a flag on and have the dep be optional, and just error if you don't have the dep when you try to use it

[botiapa]

I also have other improvement ideas such as selecting and capturing only one window (and maybe routing only its sound) and maybe using x11grab's follow argument which is meant to follow and then change the monitor that is captured depending on where the cursor is. (if I understand it right)

But I'm not sure how we could follow around an application window with ffmpeg.

[ShayBox]

There's probably a way, tomorrow I'll look into window sharing, then maybe audio, I have to figure out a way to get system sound without discord so people don't hear themselves, and maybe a way to get only one applications sound

[ShayBox]

It looks like it's not possible with ffmpeg, you can crop, but it wouldn't follow, so maybe not window capture but a crop feature, gstreamer has that functionality so maybe an opt depend for window sharing, or maybe a switch to gstreamer if it's got all the things mon2cam uses, there's also something I thought of, an option that just sets up the virtual device and then you use obs's custom ffmpeg to stream to the device

[ShayBox]

Using gstreamer would also fix some problems with v4l2 because they expect you to use gstreamer, it has options for it

[botiapa]

According to this: https://superuser.com/questions/709866/ffmpeg-x11grab-a-single-window/1378465

Gstreamer allows following a window. So yes, if we can implement everything else like it was, then we should definately switch to Gstreamer .

[ShayBox]

Okay, I'm thinking a sink that is automatically set as the default sink, you would manually set the monitor of that sink as your microphone in discord like you do Mon2Cam's webcam, then set discords Output to your headphones/speakers so discord doesn't feedback, sound good? https://github.com/toadjaune/pulseaudio-config#kinds-of-pulseaudio-devices

[ShayBox]

I have it kind of working, but pulse really doesn't make this stuff easy to do, easy concepts but linking them is impossibly complex, I ended up going through pulse module source code to find out module-virtual-source lets you pass an option to create a virtual-sink that which would then send to the sink, and for some reason it also picks up the default source of the system, so now I need to get a player to send audio from the source to the default sink so users can hear whats going through it, this is really the kind of thing users should setup with jack on their systems, linux just has to have 3 different audio systems, alsa, pulse, and jack.

[botiapa]

If this is the case then maybe we should drop this, at least for now. We don't want to introduce any latency for obvious reasons. We also shouldn't mess with pulseaudio this much. (In my opinion)

But of course you do you.

On Tue, 22 Oct 2019, 09:40 Shayne Hartford, notifications@github.com wrote:

I have it kind of working, but pulse really doesn't make this stuff easy to do, easy concepts but linking them is impossibly complex, I ended up going through pulse module source code to find out module-virtual-source lets you pass an option to create a virtual-sink that which would then send to the sink, and for some reason it also picks up the default source of the system, so now I need to get a player to send audio from the source to the default sink so users can hear whats going through it, this is really the kind of thing users should setup with jack on their systems, linux just has to have 3 different audio systems, alsa, pulse, and jack.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ShayBox/Mon2Cam/issues/7?email_source=notifications&email_token=AEYXQMO3MYAI3HLK6RKJNKTQP2U7NA5CNFSM4JBUZ5S2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEB42CGY#issuecomment-544842011, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEYXQMMIGU2624JVO7QKRD3QP2U7NANCNFSM4JBUZ5SQ .

[jamezrin]

This would be indeed a great thing to have

[botiapa]

I tried the current discord build, and it actually works pretty good, altough sound-per-application would be pretty good.

[botiapa]

@ShayBox Are you interested in continuing development?

[camdenorrb]

This would be awesome

[botiapa]

Wait do you guys still want this happen? @camdenorrb Have you tried the latest discord build?

[jamezrin]

What does the latest discord build have to do with this?

[botiapa]

I originally used this app because discord was not working for me at all. But after the release of an update it started working, and since then I hove no problem with it whatsoever. I'm just asking that, do you need this because of the extra functions, or because discord is just not working for you?

[jamezrin]

I don't understand what wasn't working for you. I use this because I cannot screenshare specific monitors, that's all. Can you now select the screen you want to share? If that's the case, what distribution are you using?

[camdenorrb]

I mainly use it to screenshare on servers stubborn to enable the live feature even though every member wants it lmao.

Also audio sharing doesn't work for me in general, so that would be a nice plus aswell

[botiapa]

@jamezrin I'm using arch and I cannot select individual screens (monitors) however I am able to select the windows themselves. My problem was, that the whole screen share thing did not work at all, what I mean is that I couldn't even share the whole screen. Sorry if I misunderstood your problem.

[botiapa]

@camdenorrb I totally agree with you. Audio sharing would be very nice.

[jamezrin]

@botiapa Now I understand what you meant. For a moment I thought, that maybe, in a distribution like Ubuntu, individual screen selection worked. I also use Arch and I have always been able to share windows and a combination of all of my monitors.

It would indeed be great if we could share the audio of the desktop through the virtual cam.

[ShayBox]

Unfortunately adding audio is over my knowledge and seems like it would be more invasive than the program should be, it requires creating a virtual output device for the system to output to, then to loop that back to your output device, then capture your output device and send it to a virtual input device, and maybe with a forking in the middle to prevent discord being sent back, it should be done with jack on the users end in reality, well it should be done by discord themselves, obs is able to capture single application output, discord could as well.

[botiapa]

Sorry for reopening, but how does obs do it then?

[ShayBox]

You're fine, I just closed since I don't plan on working on audio streaming

I'm not really sure how it does it, probably directly with alsa instead of pulse, but that's my best guess.

[botiapa]

@jamezrin If you're still interested, try out the latest commit. It's got experimental audio support.