User configures dask-ms to use processes in some fashion, possibly as a keyword to xdsfrom{ms,table} or via some global configuration option.
Instead of creating a ThreadPoolExecutor a DummyExecutor can be created.
class DummyFuture(object):
__slots__ = ("result",)
def __init__(self, result):
self.result = result
def result(self):
return self.result
class DummyExecutor(object):
def submit(self, fn, *args, **kwargs):
return DummyFuture(fn(*args, **kwargs))
Comments
As supporting processes is not currently a high priority, this feature is also not a high priority but exists to describe how it might be supported.
As an aside test_multiprocess_table shows that it's possible to spin up xds_from_ms once Processes have been forked and a fair amount of effort is taken to shutdown any threads that are started in the main Process before this happens
The problem
The solution
Comments
As supporting processes is not currently a high priority, this feature is also not a high priority but exists to describe how it might be supported.
As an aside test_multiprocess_table shows that it's possible to spin up xds_from_ms once Processes have been forked and a fair amount of effort is taken to shutdown any threads that are started in the main Process before this happens