search

daskos / epos

Write DAG Workflows like epic poems via Mesos, Marathon, Chronos, Spark, Dask
4 stars 1 forks source link

Chronos tasks w/o docker #4

Open nz87 opened 8 years ago

nz87 commented 8 years ago

We use the $MESOS_SANDBOX variable where cloudpickle should reside but when we run our stuff cloudpickle can not be found. The variable and it's content should be checked.

nz87 commented 8 years ago

After some investigation we noticed that chronos uses it's task/job id as directory name which contains colons and that directory is in $MESOS_SANDBOX which we'd liked to add to $PYTHONPATH but it will be cut by colons even if they're escaped. On the other hand inside a docker container $MESOS_SANDBOX is /mnt/mesos/sandbox: no colons = no problems.