It turns out our incompatibility with newer versions of joblib were due to mocking we were doing in the tests.
In v0.12, a joblib change meant that it relies on callback from the result object to decide when to terminate batched consumpution of inputs. Two of our tests mock the result object, which means that joblib's supplied callback doesn't get called. Set n_jobs=1 so that joblib consumes the inputs all at once and doesn't need the callbacks.
This PR also relaxs requirement on the joblib and cloudpickle version numbers, and modifies our custom backend to include some new backend features.
It turns out our incompatibility with newer versions of
joblibwere due to mocking we were doing in the tests.In v0.12, a
joblibchange meant that it relies on callback from the result object to decide when to terminate batched consumpution of inputs. Two of our tests mock the result object, which means that joblib's supplied callback doesn't get called. Setn_jobs=1so thatjoblibconsumes the inputs all at once and doesn't need the callbacks.This PR also relaxs requirement on the
joblibandcloudpickleversion numbers, and modifies our custom backend to include some new backend features.Closes #296