search

numbas / numbas-lti-provider

An LTI tool provider to run Numbas exams
https://docs.numbas.org.uk/lti/en/latest/
Other
11 stars 12 forks source link

SEB on mac: despite connecting to Numbas LTI provider, exam never loads #283

Closed Spruce8913 closed 8 months ago

Spruce8913 commented 10 months ago

In our assessment set up we have students trying a "fake" Numbas exam with Safe Exam Browser at home, so they know everything is working. Out of 180 students we had around 4 students on mac who installed SEB on mac, Numbas LTI provider seems to have accepted their SEB configuration but then the exam never loads.

Student's are stuck on the screen (within SEB): stuck on numbas loading screen

This picture is from MacBook Air 2019 (macOS 10.14.6) and Chrome opening Numbas exam. It also occurred for one student on Macbook Air (macOS High Sierra Version 10.13.4 ) with Chrome, and 2 other students with mac.

The numbas LTI provider registers they clicked the link, but labels the attempt as "not attempted". I can send you the raw data if needed. The SEB log files of a student having this issue are attached here: Some SEB log files of a student having this issue.zip

christianp commented 9 months ago

@Spruce8913 Has this happened any more since you reported this?

The log files don't seem to contain anything useful to diagnose this. It looks like the exam got stuck waiting for something to load, or with an error at some point in the loading process.

If it's intermittent, I'd guess it's something to do with a file failing to load.

Spruce8913 commented 9 months ago

This is one of the most frequent reasons we are currently forced to schedule a pc for a student on campus, rather than them using their own laptop. I have never seen a macbook successfully loading the numbas exam, and then on a separate occasion getting this loading issue. So it really seems to be an issue with the particular macbooks, rather than a temporary internet issue. Most of the laptops involved were also connected to (fast and reliable) campus wifi.

I will try to get a hold of some students having this issue, to see if we can open the developer console in Safe Exam Browser to see at what point it gets stuck.

christianp commented 9 months ago

Ah, that's at least encouraging that there will be an underlying cause we can fix.

Spruce8913 commented 9 months ago

I'm going to invite students (around 15 students) who had this issue. What would you find interesting for me to try on those laptops? I'm already definitely going to open developer console in SEB, and see what's happening networkwise.

Spruce8913 commented 9 months ago

I have found a pattern with 3 students: all of them still needed to update their macOS. After that, this issue was resolved. It thus seems to be not a Numbas issue.

It might be nice for Numbas to notify the user a certain piece of software is missing to complete, instead of getting stuck in "Loading..." but I could not get Web Inspector open in Safe Exam Browser to pinpoint where the downloading is halted. So I cannot tell you at what point the communication with Numbas LTI provider gets stuck on "Loading...".

Spruce8913 commented 8 months ago

WebInspector is currently unavailable in SEB on mac (https://github.com/SafeExamBrowser/seb-mac/issues/327) so it's not easily possible to see at what request it stops.

You might want to close this issue with concluding students need to update their mac before a numbas+seb exam.

christianp commented 8 months ago

OK, thanks.

I suspect if it's down to the version of macOS, then the problem might be in Safari as well. If you encounter another student who has this problem, could you ask them to try and load a Numbas exam in Safari and see if there's an error?

I'll close this issue for now.

Spruce8913 commented 8 months ago

If I remember correctly, we also opened a non-SEB exam with Safari. This didn't cause problems. It seems to be a quite exotic issue of an outdated mac+SEB+Numbas. I reckon you can still leave this issue closed (as SEB also does not have tools to analyse it further currently).