search

trampgeek / jobe

jobe is a server that runs small programming jobs in a variety of programming languages
MIT License
108 stars 78 forks source link

num_jobe_users=multiprocessing.cpu_count() clobber #61

Closed mkfenton closed 1 year ago

mkfenton commented 1 year ago

Howdy, it's Michael from University of Wollongong, Australia

In amongst our process to get ready for Exams Period, we spotted an issue in v1.6 that capped performance in the jump from 8 to 16 vCPUs. Thankfully, this was already spotted and corrected in v1.7 however, just flagging that commit 6bcf49f has rolled back the num_jobe_users from number of cpu's back to number in the config file only in origin/master potentially by accident?

trampgeek commented 1 year ago

Hi Michael, many thanks for posting. There is a problem here, I admit, but the commit to revert to a default of 8 CPUs wasn't actually an accident.

In commit 072a787 I reset the config file back to a fixed value of 8 because of problems for people directly running the prebuilt containerised jobeInABox. I wasn't able to find an easy solution to the problem, as the number of CPUs in /proc/cpuinfo isn't the same as the number that Docker might be allowing a container to use, and neither may match the number of CPUs on the machine on which the Docker image was built.

I reverted to using a fixed value in the config file. You can easily customise your Jobe server by setting the desired number of parallel executions in the line

$config['jobe_max_users'] = 8

in <jobe>/application/config/config.php. Then rerun the installer with the command

/var/www/html/jobe/install --purge

But ... I admit I haven't documented this anywhere except in the config file itself. Sorry!

mkfenton commented 1 year ago

hahah all good.. I figured out how to build it with 16 workers for the 16 vcpus thrown at it.. and it managed to cope very nicely with our autumn exams period (yes, apparently I'm a super-hero in the comp sci faculty meetings) just thought i'd flag in case that commit was just by accident =)

you'd be laughing at how much time I wasted scratching my head benchmarking around why we were running into errors with only 62.5% CPU usage =P ohh.. jobe_max_users was 10 (v1.6) and 62.5% of 16 is.. 10. sigh there goes a week of my life I'm not getting back.

If you're ever in Sydney or Wollongong, feel free to ping me - I'm sure there'd be a free beverage on offer

trampgeek commented 1 year ago

Congratulations on getting it all going and on achieving superhero status. But I'm certainly not laughing at your wasted week - sorry about that. Thanks for being nice about it, and even tossing in the offer of a free beverage. The same applies to you if visiting Christchurch, of course.

I'm feeling sufficiently guilty that I have now updated the documentation on both github/jobe and github/jobeinabox to explain how to set the number of Jobe users. For jobeinabox, I really should have Docker build parameter for that but I have too much on at present. As usual.