search

dask / old-dask-yarn

Deprecated, please use https://github.com/jcrist/skein or https://github.com/dask/dask-yarn instead
7 stars 2 forks source link

specifying nn and rm does not stop autodetect #9

Closed perryhook closed 7 years ago

perryhook commented 7 years ago

I'm not sure if this behavior is the desired one or not, but simply specifying nn, nn_port, rm, or rm_port does not pass this along to Knit. I'm guessing this is not desired b/c of the comment. Starting at line 47 in core.py:

# if any hdfs/yarn settings are used don't use autodetect
if autodetect or not any([nn, nn_port, rm, rm_port]):
    self.knit = Knit(autodetect=True, validate=validate)

which ignores the settings unless the default True value for autodetect is overridden. I found it very confusing that simply passing the nn and rm args were not enough to have those pass through to Knit, but I also had to add autodetect=False.

Is this the desired behavior or a bug?

quasiben commented 7 years ago

This is a bug. The intended behavior is as the comment states. I'll submit a PR to fix. Thanks!