High CPU usage when SEB is opened

[simoneeu]

Hi all, I'm trying to configure SEB 3.3.2 in my Virtual Lab that is completely virtualized. I'm using RHV for our VDI. When I open SEB without any configuration (except the one that permit me to open it on a vm) i notice that the cpu usage is very high. I have Vms with these specs:

• windows 10 ltsc
• 4gb ram
• 2vcpu The usage of CPU grow up till 60%/70% and only SEB is opened without any interaction with keyboard or mouse. This behavior could be a problem when I have many Vms running SEB at same time. Our hypervisor doesn't have as many CPUs as SEB and Vms need. I noticed that I have the same results on a virtual machine running on Vmware Esxi with same specs (cpu/ram/os).

Do you have any ideas? Thanks a lot

Simone

[dbuechel]

We'll try to have a look at it, there now seem to be quite a few users with the same issue (see #114). As a side note however, we do run hundreds of exams with 10'000+ users every year (using VMware technology) and have thus far not experienced any significant performance issues.

[simoneeu]

Thank you very much Damian. We also had hundreds of exams at same time in the past using SEB 2.4 without any problems of overload. Our hypervisor offers usually 48, 56 or 72 of logical cpu cores, but each of them virtualize 80/90/100 vm. If we have many exams in our rooms at same time, there is the possibility that the intensive use of cpus made by SEB 3.2 can create problems like "wrong countdown time" or something similar. Is there a "pointer-positioning-control" active during the execution of SEB? We noticed that an other exams portal we used had developed a real-time control of mouse pointer that caused the same behavior on a Windows VM.

[dbuechel]

I can't quite see a bottleneck at the moment, there sure are a few improvements in the real-time monitoring and also new features (like SEB-Server) in version 3.x but nothing that an average system (be it physical or virtual) can't handle. We'll have a closer look at it on our systems once we get the time to do so.

Is there a "pointer-positioning-control" active during the execution of SEB?

Yes, there is some low-level functionality like keyboard and mouse hooks, see https://github.com/SafeExamBrowser/seb-win-refactoring/tree/master/SafeExamBrowser.WindowsApi/Hooks.

[simoneeu]

Ok Damian, thank you again for fast reply. I know that SEB 3.x can run on average system, but my problem is when too many SEB 3.x are opened at same time on a hypervisor that hosts many Vms. I will look at keyb and mouse hooks you wrote, thank for the tip!

[forobliot]

we made a lot of debug testing trying to figure out where the code lead to a cpu performance issue and we found this lines of code in HandleSessionStartSuccess method of RuntimeController class (SafeExamBrowser.Runtime component):

` RegisterSessionEvents();

                                            runtimeWindow.ShowProgressBar = false;
                                            runtimeWindow.ShowLog = Session.Settings.Security.AllowApplicationLogAccess;
                                            runtimeWindow.TopMost = Session.Settings.Security.KioskMode != KioskMode.None;
                                            runtimeWindow.UpdateStatus(TextKey.RuntimeWindow_ApplicationRunning);

`

It happens that if we set ShowProgressBar to true the cpu usage become normal

we tested with success this code on a simple vm in our vmware infrastructure and Simone tested on RHV, but we can't evaluate if this change in the code can affect the behavior of the browser. Can you check if this change affects the safe browsing?

thanks ale

[dbuechel]

Excellent work, thanks for investigating the issue further @forobliot! It must then be that there is a performance issue in WPF, specifically with animated storyboards. Setting the progress bar to visible will hide the animated border of the runtime window: https://github.com/SafeExamBrowser/seb-win-refactoring/blob/master/SafeExamBrowser.UserInterface.Desktop/Windows/RuntimeWindow.xaml.cs#L43

The animated border itself is some eye candy which is there since the very first version of SEB 3.x. The idea behind it was to visualize that a session is running (as opposed to when the progress bar is visible, which indicates that some initialization or finalization work is being performed): https://github.com/SafeExamBrowser/seb-win-refactoring/blob/master/SafeExamBrowser.UserInterface.Desktop/Windows/RuntimeWindow.xaml#L18

I'll try to measure the performance impact, see whether I can optimize something related to the animation or, in the worst case, simply remove it altogether as the runtime window normally isn't visible to the users once a session is running.

[dbuechel]

I have completely removed the animated border. This gives me on my machine a (roughly measured) performance improvement of 1-3%.

@forobliot Most unfortunately, our build server isn't working properly at the moment. If possible, could you pull the latest changes and verify whether this change indeed improves the performance in your virtual environment?

[forobliot]

Hi Damian,

I tested the new modified version and it is ok on my test vm in vmware Simone will check soon on RHV environment

thank you

[dbuechel]

Excellent, glad to know that @forobliot. Out of curiosity, would you mind specifying how big the performance improvement is? I'm still a bit in awe that this animated border caused any issue at all...

[forobliot]

from 90% to 1-2% in my test vm. I've compiled an "homemade" setup bundle from your code, I hope it is correct. It shows a v1.0.0.0 version in the starting splash screen

[dbuechel]

from 90% to 1-2% in my test vm

Okay, that's just crazy! Maybe it could have something to do with graphics performance in virtual machines?! Incredible.

I've compiled an "homemade" setup bundle from your code, I hope it is correct. It shows a v1.0.0.0 version in the starting splash screen

Yes exactly, local builds always have the version number 1.0.0.0. We however just were able to fix our build server, so you can now test the change in the latest beta build: https://sebdev-let.ethz.ch/api/buildjobs/the9xs6xy0lohx3u/artifacts/SEB_3.4.0.427_SetupBundle.exe

[forobliot]

Ok, I tested this official beta version and still no performance issues I am waiting for the response from the RHV test and if low cpu usage is confirmed we will start using this beta in our labs until final version 3.4 is available: exam.net will drop support of seb v2.4 version very soon

[dbuechel]

Great, that's good to know. Regarding the usage of a beta version: We strongly advise against that, as this is merely a random development build which has not yet been tested internally. So you'd be using this build at your own risk...

[dbuechel]

Any updates here @simoneeu / @forobliot ?

[forobliot]

the test with RHV was ok. We installed this version in our labs, we are aware that it is a beta version

thank you

[dbuechel]

A fix for the issue is now available in version 3.4.0.