search

Thinstation / thinstation

A framework for making thin and light Linux based images for x86 based machines and thinclients.
https://www.thinstation.net/
797 stars 187 forks source link

Segfault when accessing webcam via Citrix client #607

Closed justinchevrier closed 3 years ago

justinchevrier commented 3 years ago

Hey Guys,

We're experiencing the following segfault when accessing a webcam via the Citrix client (accessed via Firefox from within the Citrix session):

v4l2src:src[10557]: segfault at 6561616a ip 00007f5bc69b14e3 sp 00007f5bc5cbcbd0 error 4 in libglib-2.0.so.0[7f5bc697e000+85000] Code: 8b 30 48 c7 46 08 00 00 00 00 4c 89 60 08 48 8b 35 72 bc 0d 00 49 89 c4 4c 01 f6 48 8b 1e 48 39 d8 0f 84 80 00 00 00 48 89 c8 <48> 8b 38 41 89 d0 48 8b 77 08 89 f1 41 29 f0 29 d1 39 f2 49 0f 47

The browser within Citrix will show a few frames (I'd 1 - 3 frames) from the webcam before freezing.

Webcam is a Logitech C270: Bus 001 Device 007: ID 046d:0825 Logitech, Inc. Webcam C270

Git log: commit c4d1700a115f81c2165e35d4628c3c56155c87bd (HEAD -> 6.2-Stable, origin/HEAD, origin/6.2-Stable)

Thanks.

Thinstation commented 3 years ago

Which version of Citrix? Is this in a test VM?

justinchevrier commented 3 years ago

Citrix client version is: 20.04.0.21 but have also tried: 20.9.0.15 with same result.

Citrix server version is: 1912 CU1

Thinstation is running on: HP T520 - HP t520 Flexible Series TC (SKU Number: G9F06AT#ABA)

same segault also happens on: HP T530 - HP t530 Thin Client (SKU Number: 3GM98UT#ABA)

Thinstation commented 3 years ago

I tried bumping glib to 2.66.2 from .0. There were several reported bug fixes, but I can't test your full scenario. Let me know.

Thinstation commented 3 years ago

Possibly this is gstreamer. Looks like ICA supports 1.x branch now. I'll try and enable it.

Thinstation commented 3 years ago

Also looks like missing llvm libs for webrtc. That could be it as well.

justinchevrier commented 3 years ago

Thanks! I appreciate your efforts!

I pulled commit faad919356060eb6522720ee18da516ef1b49f40 and tested again.

I'm no longer getting the segfault, but the camera still only works for a frame or 2 and then stops (the LED which indicates camera is in use also shuts off).

Interestingly, I have the Zoom VDI plugin for Citrix installed and the camera works through the Zoom client using the plugin.

In Thinstation 6.1 (commit e292e3b4178c1af7816b64f7cfc970b7b4b8ec2a) the camera works fine (even when using the latest (20.9.0.15) Citrix client).

Thinstation commented 3 years ago

There were definitely unresolved library issues. I've taken that pipeline to the current release of all involved libs. Give it a go, let me know.

justinchevrier commented 3 years ago

Still not working unfortunately. The same segfault is there as well, but it doesn't appear consistently.

Thinstation commented 3 years ago

Can you send your build.log

justinchevrier commented 3 years ago

Just to confirm, is that just a redirect of the build process? eg. ./build &> build.log

Doncuppjr commented 3 years ago

I use stb, which puts a log in the root of chroot

On Tuesday, October 27, 2020, 10:54:30 AM PDT, justinchevrier notifications@github.com wrote:

Just to confirm, is that just a redirect of the build process? eg. ./build &> build.log

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe .

justinchevrier commented 3 years ago

Build log is attached.

build.log

Thinstation commented 3 years ago

Well all that looks good. What is your full test scenario again? Which version of Firefox? On windows? Have you tried Chrome?

justinchevrier commented 3 years ago

Firefox ESR 78.4.0 on Windows Server 2016. I haven't tried Chrome, but can certainly give that a try.

Testing procedure is; once logged onto Windows desktop via Citrix, launch Firefox and then access a webpage that tries to utilize the webcam in the browser (for example: https://www.onlinemictest.com/webcam-test/ )

Thinstation commented 3 years ago

I don't have an ICA test platform, so I can't do the tests myself, but you could make a strace -f -tt -o /debug -p and send it to me, I'll take a look. You add strace to your build. Once booted, make the ICA connection, open a shell, su, attach to the pid, then access the website that goes boom.

Thinstation commented 3 years ago

I found a support page that says ICA can only do content redirection with IE and Chrome.....Nov. 2017, so not very current, but try chrome.

justinchevrier commented 3 years ago

Will do. In the meantime I did grab an strace, but of course haven't been able to get the segfault again after multiple attempts. Is there any value in taking a look at it when capturing over the time of trying to access the camera even without a crash? If so, it's 69MB on compressed, should I just compress and attach here? Thanks.

Thinstation commented 3 years ago

Wow, big, might need to share via google drive.

Thinstation commented 3 years ago

Found this. https://docs.citrix.com/en-us/citrix-virtual-apps-desktops/multimedia/webcam-compression.html

justinchevrier commented 3 years ago

Yes, we are using HDX Webcam Redirection (versus sharing it as a USB device). According to the following link it only works in 32-bit applications in the VDA when using the Linux Workspace App, but we are in fact using a 32-bit version of Firefox:

https://support.citrix.com/article/CTX132764

I sent you an email with the strace. Hopefully I can do a test in Chrome tomorrow.

Thinstation commented 3 years ago

Looks like server uses mediafoundation, and firefox is still using directshow......idk

justinchevrier commented 3 years ago

I ran another test in Thinstation 6.1 and in that version the webcam works fine in Firefox (I'll have to wait until tomorrow to test Chrome). I ran strace while successfully accessing the webcam and the file is significantly smaller at around 7MB.

Thinstation commented 3 years ago

That answers some questions.....now, was that in hdx mode or usb passthrough?

justinchevrier commented 3 years ago

HDX mode.

Thinstation commented 3 years ago

Using a tsuser or root?

justinchevrier commented 3 years ago

root in both cases

Thinstation commented 3 years ago

Ok. So what if you install ICA in the chroot environment and launch from there?

Thinstation commented 3 years ago

It doesn't even look like 20.9.0.15 installs on 6.1. How are you testing?

justinchevrier commented 3 years ago

Building with 20.9.0.15 fails on 6.1 for you or it builds but won't run afterwards?

Doncuppjr commented 3 years ago

It fails. Missing webkitgtk.....

justinchevrier commented 3 years ago

How do I run ica from within the chroot environment?

Doncuppjr commented 3 years ago

Just go into the chroot, then act like you were going to install it on a typical distro. Unpack and run the install. Launch

justinchevrier commented 3 years ago

I tried running it in the chroot but before it shows the desktop it throws up these error messages as dialog boxes but also logged in the console:

30203: stderr from child

The X Request 130.1 caused error: "10: BadAccess (attempt to access private resource denied)".

End of stderr (96 read)

30203: stderr from child

The X Request 130.3 caused error: "128: BadShmSeg (invalid shared segment parameter)".

End of stderr (88 read)

Thinstation commented 3 years ago

What host distro are you using?

justinchevrier commented 3 years ago

Gentoo

Thinstation commented 3 years ago

Is it a container?

justinchevrier commented 3 years ago

No, on bare metal

Thinstation commented 3 years ago

Bummer security issue. Works on Fedora, well at least it starts.

Doncuppjr commented 3 years ago

I added more missing libs. Maybe it works now.

justinchevrier commented 3 years ago

I tried again. No change unfortunately. I believe that I managed to grab an strace of the segfault however, but I'm not sure how useful it will be. I will email it to you. It is significantly smaller than the previous one at 900KB. I had to wait until right before it was going to display the camera image otherwise it never seemed to segfault when strace was attached to the process.

I was doing some reading about debugging segfaults and came across this article: https://jvns.ca/blog/2018/04/28/debugging-a-segfault-on-linux/

It talks about setting a core dump output using: "sysctl -w kernel.core_pattern" but I am unable to configure that setting and get the following error:

sysctl: error: 'kernel.core_pattern' is an unknown key

Would it be possible to include that in the Thinstation Kernel?

Thinstation commented 3 years ago

Yes. I don't normally include that option, but you could add any kernel config options your want to ts.config, and then run rebuild-kernels -a

Thinstation commented 3 years ago

Your gentoo install have X?

justinchevrier commented 3 years ago

Okay, I'll check that out thanks. Yes I do have X on my Gentoo system.

Thinstation commented 3 years ago

I ask, because troubleshooting this problem would be a lot easier if you didn't have to make a new build for every test. Possibly your system is using private tmp files, and that's not letting apps inside the chroot connect with X.

justinchevrier commented 3 years ago

I compiled that option into the kernel and configure the kernel.core_pattern parameter now, but it's not dumping the core.

I did notice the following in dmesg however: This one happens when there's a segfault: _audit: type=1701 audit(1604439823.654:70): auid=0 uid=0 gid=0 ses=1 pid=14000 comm="v4l2src:src" exe="/opt/Citrix/ICAClient/util/gstread1.0" sig=11 res=1

This one happens when there isn't a segfault (but the camera still only shows a single frame): _audit: type=1701 audit(1604439765.009:69): auid=0 uid=0 gid=0 ses=1 pid=13280 comm="v4l2src:src" exe="/opt/Citrix/ICAClient/util/gstread1.0" sig=6 res=1

justinchevrier commented 3 years ago

Ya, a good idea for sure. I'll put some effort into determining the cause of the SHM permissions issues.

Thinstation commented 3 years ago

I do hope that we can work this. Possibly we could force this back to gstreamer 0.10. It's old, but maybe that works.

justinchevrier commented 3 years ago

Here's some more information:

"_gstread | /opt/Citrix/ICAClient/util | HDX RealTime Webcam Video Compression requires GStreamer 0.10.25 (or a later 0.10.x version), including the distribution's "plugins-good" package; or GStreamer 1.0 (or a later 1.x version), including the distribution’s “plugins-base,” “plugins-good,” “plugins-bad,” “plugins-ugly,” and “gstreamer-libav” packages."

from: https://developer-docs.citrix.com/projects/workspace-app-for-linux-oem-guide/en/latest/reference-information/

Thinstation commented 3 years ago

That's a lot of plugins.......

Thinstation commented 3 years ago

All plugins added.