Create ContainJobs

[MarcSkovMadsen]

With ContainDS I am very close to being able to ditch Azure Pipelines deployments for my analyst team. But a few things are holding me back.

One of the things is that we need to be able to run python scripts on a schedule. A single user can of course open a terminal and schedule a cronjob. But a much nicer solution would be to be able to configure it in exactly the same way as ContainDS enables. And then later be able to edit or delete the job.

I would like multiple administrators enabled for a job similarly to https://github.com/ideonate/cdsdashboards/issues/69

[MarcSkovMadsen]

Inspiration for an extended interface could be found in https://github.com/alseambusher/crontab-ui

[danlester]

Thanks for all your recent ideas and contributions. I may get time (or funding) for some of these over the next few weeks. I'm keen to find a sustainable model for future development in this whole area. It's currently fine to use for a company that has someone like you to glue the pieces together, but for most organizations the 'use some free code on the internet' isn't enough to make a long term investment in using it.

One thing I've been discussing with a few users is the possibility of building a separate 'ContainDS Deploy' product for easily-configured dashboards specified in YAML in a git repo, deployed straight into Kubernetes and behind an enterprise-level authenticator. It would probably be a commercial product, at least to start with for corporates, as so much of the setup (especially authentication) really needs detailed configuration to be viable for an enterprise.

The ContainJobs (CronDS...?!) could fit into that neatly perhaps.

Also, have you seen NaaS scheduler?

[MarcSkovMadsen]

I have seen Naas. Some problems I see

• My Jupyterlab instance closes down when I close down my browser. So where would this be running permanently?
• There is no web interface to manage the jobs (create, read, update and delete).
• There is no way to get alerts or see logs

[MarcSkovMadsen]

"We" (the advanced developers) can already deploy cronjobs to Kubernetes using Azure Pipelines and .yaml files. The problem is the analysts etc. working on the Jupyter Hub. They need an integrated, self-service environment with a UI inside a familiar platform.

[MarcSkovMadsen]

For a short term solution we will be trying out adding crontab-ui as an extra launcher. So far it looks like

c.CDSDashboardsConfig.extra_presentation_types = ['crontab-ui']
c.VariableMixin.extra_presentation_launchers = {
    'crontab-ui': {
        'args': ['crontab-ui'],
        'debug_args': [],
        'env': { # should these environment variables be set in args instead ??
            'PORT': '{port}',
            'BASE_URL': '{base_url}', # origin_host, or ???? It should only be the /mt-workspace/user/masma/APP-NAME part
        }
    }
}

I will post the working configuration when we get it tested and working.

https://github.com/alseambusher/crontab-ui

[danlester]

Sounds interesting! That looks about right to me as far as I know...