[Inquiry] Can Datahub Be Used For Large Class-Size Examinations?

[cdbeon]

PH 142 is currently looking into how feasible it is to hold our final exam online on Datahub. Our current plan is the following:

1. Send out a compressed file containing the exam Rmd as well as a dataset in csv format
2. Students will simultaneously complete using the data provided. This will be done on Datahub (or local RStudios, if they choose to do so)
3. Students will knit their file and submit their exam onto Gradescope

A few additional details/logistics about the course and exam:

• We currently have 340 enrolled students
• The exam is set to take place on Monday, December 13th from 7-10 PM (3 hour duration)
• This class has been using the Public Health hub since the beginning of this semester (Fall 2021)
• The size of the dataset is projected to be less than 40K at most.
• The most computationally intensive task the students will be to do is run a permutation test (up to 10,000 permutations, can scale down if needed)

Given this information, would it be possible to hold said exam with the current resources available? Would be happy to provide more information if needed.

[cdbeon]

@corinne-riddell @kmaccuish

[balajialg]

Thanks, @cdbeon for sharing the detailed requirement! Posting slack response below for the sake of continuity

I guess provisioning more nodes during 3 hours of the exam is something we can definitely do at our end. We did try the synchronous use case with the Political Science 3 course, where 300+ students worked on R notebooks during the class hours. We had some hiccups initially but were able to make it work through few workarounds.

Let's discuss the way forward in this thread!

[felder]

@cdbeon This sounds like something we should totally be able to accommodate using the scheduling calendar to increase node provisioning during this period or perhaps from 6-11 just to give a bit more wiggle room.

It may also be helpful if students are encouraged to sign in a little ahead of time (if they can) so we don't have 340 folks all hop on at once at 7pm. However, I think if we go for the scheduled increased node provisioning route it'll probably still be ok.

Also you mention the students use publichealth hub currently, but that the exam would take place on datahub? Is this right?

@yuvipanda do you think maybe we should also provision more cpu for each student during the exam period just to make sure there are no permutation hiccups?

[cdbeon]

Forgive me, I thought Datahub was just the catch-all term for all the hubs.

This exam would take place on the Public Health Hub.

[felder]

@cdbeon just wait until the "Datahub" building is constructed!

https://statistics.berkeley.edu/about/news/largest-gift-berkeleys-history-data-hub-building

[ryanlovett]

@felder Fortunately it is now called the Gateway Building. I guess that creates an issue for Network Operations however. :)

[yuvipanda]

@felder Someone should run through the expected notebooks on datahub and see CPU / memory usage. We need to increase CPU limits only if they are planning on using a lot of CPU

[balajialg]

@cdbeon Whenever the teaching team finalized the decision wrt Datahub and are ready with the notebooks, Can you share the notebooks with us for evaluating CPU/RAM requirements? Few days' heads up prior to 12/13 would be appreciated.

[felder]

@balajialg @yuvipanda I've added an entry into the scaling calendar for 12/13 from 6-11 and specified 6 extra nodes in the beta pool for this.

I also moved the evening cooldown entry to after 11.

[balajialg]

@felder Thanks for looking into it! Did you get any confirmation from the teaching team or @cdbeon that they will use the hub for their exams?

[felder]

@balajialg no

[balajialg]

@cdbeon ASAp, Can you confirm whether your plan to use datahub as part of this examination is in scope? If not, we can revert back to the original setting

[cdbeon]

Hello! Sorry about the delay, yes we will be using Datahub as part of the exam. However, the amount of computation that the students are required to do will be much lower than we originally expected.

[balajialg]

@cdbeon That's good to know! Let's keep @felder's allocated compute as is during the exam time to ensure that the odds of the hub struggling to scale up is pretty low. Thanks!

[felder]

@cdbeon how did the exam go last night?

[cdbeon]

It went smoothly! No complaints about the hub crashing or lagging. Thank you all so much!

[felder]

Great! With that, I'm going to close this issue.