SafeExamBrowser / seb-mac

Safe Exam Browser for macOS and iOS
https://www.safeexambrowser.org/macosx
96 stars 39 forks source link

Error when loosing internet connection with Moodle #356

Open lionelasshauer opened 5 months ago

lionelasshauer commented 5 months ago

Describe the bug Environment: Moodle Quiz using the template setting / uploaded configuration - tested with different SEB configuration files.

When SEB loses the internet connection during an exam (e.g. WiFi issues) on Windows, loading a page will simply "pause" and resume as soon as the connection is restored. When losing the connection on a Mac, SEB will report a connection error, allowing to abort or retry. After restoring the connection, instead of properly loading back into the exam, we end up on a Moodle error page reporting that the parameter “attempt” is missing.

Using Sonoma (AAC and non AAC) & Big Sur (non AAC) The error page allows me to get back to the Moodle Dashboard, leaving the secure quiz environment (which we do not want). It is possible to reenter the exam.

Using Big Sur (AAC with pre pinning enabled) The same error page displays, but clicking on any links on that page will lead to a connection error, even though the device is connected and was able to load the error page.

Would it be possible to achieve the same behavior as on Windows, allowing students to resume taking the quiz without leading to an error page? We are currently planning to roll out a BYOD configuration, where a connection loss during the exam is more likely to occur than in our wired Windows labs.

To Reproduce Steps to reproduce the behavior:

  1. Start a Moodle quiz using a SEB template or pre-uploaded configuration
  2. Start the quiz
  3. Disconnect the computer from the internet (tested with unplugging the LAN connection)
  4. Try moving to the next quiz page
  5. Reconnect the cable and wait a couple of seconds for the connection to re-establish
  6. Retry

Depending on system and AAC

Expected behavior Same as on Windows, allowing me to resume the exam and saving inputs that were entered during the time the system was offline. In addition, #342 could help to reconnect the device.

Screenshots Moodle error (same on all tested configurations)

SCR-20231222-kebe

Version Information Please complete the following information:

logs_connection.zip

lionelasshauer commented 5 months ago

Same issue occurs when Moodle is set to "Use client config"

danschlet commented 5 months ago

It's definitely a Moodle bug that

Please report that one on moodle.org and post the MDL number here. Especially it should under no circumstance happen (regardless what kind of connection error occurs) that Moodle would leave the quiz!

Regarding the reconnection behavior of SEB for macOS we will investigate. The different behavior of the Windows and Mac/iOS versions of SEB are likely caused by the different browser engines (and this Moodle bug might only be happening with WebKit). Nevertheless we never before observed that the connection error alert caused such issues (also not with previous Moodle versions).

danschlet commented 5 months ago

PS: This doesn't seem to happen when SEB can use the modern WebView. This is the case when Moodle 4.1 or newer or latest 4.0 versions are used or if you manually update the SEB integration plugin.

lionelasshauer commented 4 months ago

Hi @danschlet, I just checked our configuration. My initial test were done using a configuration where browserWindowWebView was set to 2. After changing it to 3 - which should always use the modern webview - the behavior was exactly the same.

Should I file this as Moodle Bug?

We will update to the newest Moodle version sometime next month, as of now I can only test using Moodle 4.2.2.

Corresponding log attached. SEB_BigSur_AAC_WebView3.log

danschlet commented 4 months ago

browserWindowWebView was set to 2. After changing it to 3 - which should always use the modern webview - the behavior was exactly the same.

If you use the content filtering setting for URL filters, still the classic WebView is used. Please try if pinch-to-zoom on the trackpad works, if yes, the modern WebView is used (unless you disabled zooming completely in your settings). But in your log file, "Opening modern WebView" is listed, so it seems to be the case.

I wonder why my Moodle 4.1.x behaves differently. Are you using the WiFi Resilience Plugin or another plugin which would provide some special behavior when the internet connection fails during the exam?

If you can provide me with a testing access to your Moodle, I would try to analyze the issue further. I need a reliable way how to reproduce it.

lionelasshauer commented 4 months ago

Yes Pinch-to-Zoom is working.

I'm currently checking if I can get you a test login. If possible, I will send it via mail. (I just noticed that non-sso logins I'd normally use won't work as SEB will instantly redirect to the sso page)

danschlet commented 4 months ago

Yes, you can send it via mail (if you know mine or otherwise info at safeexambrowser dot org, which only the core SEB team reads).

lionelasshauer commented 4 months ago

Hi @danschlet you should've gotten an email with the credentials.

lionelasshauer commented 3 months ago

Last week, we updated Moodle to version 4.3.3+. Unfortunately, the problem persists and we still encounter the same issues, even when we opt for using the modern WebView.

McDowell-CSSD commented 2 months ago

We encounter the same issue here with Moodle 4.3.1 and our Mac’s on Sonoma 14.3. A disconnect (or any error, in our case a student spamming the “check code” in a Coderunner type question) leads to Moodle error and this exits the secure environment.

danschlet commented 2 months ago

Maybe this is an issue in Moodle >4.1. In Moodle 4.1 I just cannot reproduce this (when the modern WebView is used). @lionelasshauer: Unfortunately your demo login doesn't work, see the mail I sent you.

danschlet commented 2 months ago

Ok, so this is a multifaceted issue, I'll explain everything later. BUT, as I already assumed previously, the gravest bug is in Moodle, probably in the SEB (Deeper) integration:

When previewing a quiz in Safari, disconnecting the internet connection and trying to click something (or when a script is trying to connect to a server in the background) leads to the typical connection error page (empty page only with text "You Are Not Connected to the Internet – This page can't be displayed because your computer is currently offline.").

When reconnecting to the internet and after waiting a bit (until Safari realizes there is a connection again), the same Moodle error message "A required parameter (attempt) was missing" is displayed. So this is obviously not an SEB bug when it also happens in Safari!

SEB doesn't display such a "not connected" page like Safari, because we would need to implement this ourselves, together with the code which tries to reconnect (tries to repeat the failed HTTP request) when the internet connection is restored. So SEB only displays the alert window indicating the connection failed error and allowing to retry manually or canceling loading the request. If the user cancels, then they need to manually reload after the internet connection was restored. This is of course only possible when Reload for the exam window isn't disabled in SEB settings (if only the Reload button isn't displayed, you can still use the common shortcut cmd-R).

Finally I was able to reproduce this Moodle error with SEB, looks like I previously always canceled the connection error alert and then used the reload button after restoring the internet connection, which then doesn't invoke that error in Moodle. Maybe I can as a workaround use the same reload mechanism also in the alert. This might then at least prevent the Moodle error happening inside SEB.

Now a further aspect might come in play when AAC is used on macOS < 14.3. There was a bug in previous macOS versions where AAC blocked the system process (daemon) which is responsible for renewing DHCP leases (see below). I guess this is the reason that even when reconnecting the internet, reload while using AAC on macOS versions prior to 14.3 doesn't work and you need to quit SEB (I observed that). It could also be that some other process is blocked by AAC which is required to reconnect to the internet in case you plug a LAN connection (or USB-C LAN adapter). On macOS >= 14.3 even in AAC the Moodle error may happen, same like in Safari.

The first thing which must happen now is to report this Moodle quiz (or Moodle SEB integration quiz) bug in the Moodle tracker on moodle.org. The bug might be related to the used browser engine, as you didn't observe it on Windows. SEB for macOS/iOS used WebKit and not Chromium.

But whenever you observe a bug in SEB for Mac and not in Windows, the first test should be to try it also in Safari. If it happens in Safari as well, then it's a Moodle bug and not a SEB bug.

See https://developer.apple.com/documentation/macos-release-notes/macos-14_3-release-notes

In macOS 14.3, an issue was fixed in which a Wi-Fi network’s DHCP (Dynamic Host Configuration Protocol) lease could not be renewed during AAC. These are dynamically leased IP addresses, and the process that took care of the DHCP lease renewal did not have internet access during AAC until macOS 14.3. This means the test will lose internet access if the lease expires during the test. When the student emerges from the test, they will see a line through their Wi-FI bars (indicating that the Mac is connected to Wi-Fi but no valid IP address) — this will quickly be restored when the lease is renewed.

Some school Wi-Fi networks have very short lease expiry times, like 2 hours. (The default is generally 24 hours, so home wifi networks will not likely see this — it’s only if a network has a lot of devices and they are worried about running out of IP addresses when they make the lease time very short.) The lease would begin as soon as the student attached to the Wi-Fi network that day. So even if your test is just 30 mins, it’s possible that the lease could expire during it if they attached first thing in the morning.

danschlet commented 2 months ago

Regarding automatic reconnection to the internet after loosing the connection: I'll create a new feature issue for that. But as this won't solve this Moodle bug, this won't be prioritized as a critical bug fix and will take some time to get onto the roadmap.

danschlet commented 2 months ago

Ok, the "A required parameter (attempt) was missing" error even happens when "Require the use of Safe Exam Browser" is set to NO in quiz settings. Therefore this really is a Moodle bug or incompatibility with WebKit browsers. Definitely needs to be reported at moodle.org.

danschlet commented 2 months ago

PS: One more finding: If I'm on a quiz page containing an essay question, the autosave feature prevents the missing attempt bug, if it kicks in before navigating in the quiz. It displays the light red popup at the top of the window. Upon restoring the internet connection, the light green popup shows the message, that saving was successful again.

So it looks like the bug can be prevented in Moodle. I would hope that the Moodle WiFi Resilience Plugin would also prevent the error to happen.

lionelasshauer commented 2 months ago

Thank you @danschlet for investigating further (hopefully I can get the demo account working on tuesday)! Could you report the bug in the moodle tracker, as you probably have a better understanding as to what exactly might be going on behind the scenes?

McDowell-CSSD commented 3 weeks ago

Has this been reported on tracker.moodle.org? I can’t seem to find it. If it has been, can someone link it here please?

danschlet commented 2 weeks ago

I haven't reported it. Feel free to create an issue on tracker.moodle.org.

danschlet commented 6 days ago

In SEB 3.3.3 for macOS is a new setting in the Settings / Browser Pane / Reload page after connection error. If you set that in the settings you're using for your Moodle quiz ("Require Safe Exam Browser – Yes, upload my own config"), in case of a load error, the Retry button reloads the complete page when the internet connection is back, so the attempt error in Moodle doesn't happen. BUT all unsaved changes on that question will be lost (use autosave for essay question types or the ETH WiFi Resilience plugin!).

Please test and report if it works for you.

Screenshot 2024-05-26 at 17 39 16
danschlet commented 6 days ago

PS: I reported the issue to our Moodle guys and they are working on having a fix in Moodle at some point.

swz23 commented 6 days ago

In SEB 3.3.3 for macOS is a new setting in the Settings / Browser Pane / Reload page after connection error. If you set that in the settings you're using for your Moodle quiz ("Require Safe Exam Browser – Yes, upload my own config"), in case of a load error, the Retry button reloads the complete page when the internet connection is back, so the attempt error in Moodle doesn't happen. BUT all unsaved changes on that question will be lost (use autosave for essay question types or the ETH WiFi Resilience plugin!).

Please test and report if it works for you.

->

I only see "Reload page after connection error" when I switch macOS to English. The German version still says "Java aktivieren" next to this checkbox in 3.3.3.

danschlet commented 5 days ago

That's the same setting, this is a localization mistake. Thanks for reporting it, I will fix that.

danschlet commented 3 days ago

The localization issue for the setting "Reload page after connection error" is now fixed in the final release version of SEB 3.3.3 (build 151C0).