Public-Health-Scotland / posit-workbench-job-scheduler

Functions to assist with programmatically launching Workbench Jobs on the Kubernetes cluster, and scheduling these to run a specific time.
4 stars 2 forks source link

Allow group of users to test #15

Open terrymclaughlin opened 1 year ago

terrymclaughlin commented 1 year ago

@Public-Health-Scotland/posit-workbench-job-scheduler-team

I've set up a few users with read access to this repo for testing purposes. I've been trying to keep this testing group limited, and I've not had any feedback from them yet on whether it's working for them or not.

Whilst I'm on leave, please just add other users to this repo so that they can be involved in testing, then we can review again on my return whether we open this up for general use.

Thanks!

daveb-phs commented 1 year ago

Hi @terrymclaughlin,

Most of the scripts we currently schedule use a keyring to access one of the databases. I've tried the launch_workbench_job() function but it's not working with the keyring. Like @jakeybob I'm getting a "Permission denied" error until I comment out the 'container' argument in the launcherSubmitJob() command.

Having done that my scripts fail with the error:

Error in b_file_keyring_unlock(self, private, keyring, password) : Keyring `` does not exist Calls: -> -> b_file_keyring_unlock Execution halted

This doesn't happen if I just run the script as a workbench job through POSIT Workbench's dialogue box. I notice that if I add Sys.info()[["user"]] to the script then when I launch the job with launch_workbench_job() it returns user "root", but when I run the workbench job through POSIT's dialogue box the username returned is "unknown".

The script completes successfully when run through the dialogue box, so I'm thinking that the keyring is not being found because the job created by launch_workbench_job() is running as "root".

I tried adding the "user" argument to launcherSubmitJob() but it didn't make any difference.

jakeybob commented 1 year ago

Yep this ties in with what I was seeing. It's definitely not running with root permissions (at one point I tested a job to see if it would list files in directories I don't have access to, and it wouldn't) but with the container line commented out it is running as some sort of stripped-down root, and not as the user who starts the job. I don't really understand why different people are seeing different behaviours.

Running the same stuff via the workbench GUI seems to work fine, and as far as I can tell it launches with the same parameters/arguments etc 🤷🏻 . Part of me thinks it might be to do with renv sourcing its own .Rprofile ... or maybe even a disk permissions thing. Will try to start from a clean project and look into more systematically!

Theresa2020 commented 1 year ago

Hi Terry, just want to say that I've been managing to do some automations - scheduling them from the day before with NIGHT at the start of the session name. I tried to do a few automations for the same time on Tuesday, but with the other issues with Posit only one kicked off at 9am when the issue was fixed. Should we be able to schedule a few automations from the same session? Can I ask that one of my staff members is also given access so that he can test also, his username is mark-hamilton.

terrymclaughlin commented 1 year ago

@mark-hamilton now has access.

terrymclaughlin commented 1 year ago

@Theresa2020 Yes, you can (in other words, should be able to) set up multiple automations / schedules from the same session. So if you have 3 separate scripts that need to run at different times, they can be set up on three separate schedules from the same scheduler session.

Theresa2020 commented 1 year ago

Thanks Terry, what happens if you need them to be scheduled at the same time? Should this in theory still work?

terrymclaughlin commented 1 year ago

The implementation uses the {later} package which will try to launch a Workbench Job and run your script at the time you've requested. However, it makes no real promise about this, and it is dependent on what else is happening on the CPU at that time. So, you could set up 3 schedules, all to start at the same time, but you might find that they start at slightly different times from one another (i.e. a few seconds apart).

Theresa2020 commented 1 year ago

Great, thanks Terry.