search

rapidsai / xgboost-conda

Conda recipes for xgboost
12 stars 22 forks source link

[BUG] E2E: IndexError list index out of range on dxgb_gpu.train(...) #11

Open starlett3 opened 5 years ago

starlett3 commented 5 years ago

I'm trying to run notebooks/mortgage/E2E.ipynb and failing on box 22: bst = dxgb_gpu.train(client, dxgb_gpu_params, gpu_dfs, labels, num_boost_round=dxgb_gpu_params['nround'])

Specifically, i'm getting error: IndexError: list index out of range

IndexError Traceback (most recent call last)

in /conda/envs/rapids/lib/python3.6/site-packages/dask_xgboost-0.1.5-py3.6.egg/dask_xgboost/core.py in train(client, params, data, labels, dmatrix_kwargs, **kwargs) 229 """ 230 return client.sync(_train, client, params, data, --> 231 labels, dmatrix_kwargs, **kwargs) 232 233 /conda/envs/rapids/lib/python3.6/site-packages/distributed/client.py in sync(self, func, *args, **kwargs) 673 return future 674 else: --> 675 return sync(self.loop, func, *args, **kwargs) 676 677 def __repr__(self): /conda/envs/rapids/lib/python3.6/site-packages/distributed/utils.py in sync(loop, func, *args, **kwargs) 275 e.wait(10) 276 if error[0]: --> 277 six.reraise(*error[0]) 278 else: 279 return result[0] /conda/envs/rapids/lib/python3.6/site-packages/six.py in reraise(tp, value, tb) 691 if value.__traceback__ is not tb: 692 raise value.with_traceback(tb) --> 693 raise value 694 finally: 695 value = None /conda/envs/rapids/lib/python3.6/site-packages/distributed/utils.py in f() 260 if timeout is not None: 261 future = gen.with_timeout(timedelta(seconds=timeout), future) --> 262 result[0] = yield future 263 except Exception as exc: 264 error[0] = sys.exc_info() /conda/envs/rapids/lib/python3.6/site-packages/tornado/gen.py in run(self) 1131 1132 try: -> 1133 value = future.result() 1134 except Exception: 1135 self.had_exception = True /conda/envs/rapids/lib/python3.6/site-packages/tornado/gen.py in wrapper(*args, **kwargs) 324 try: 325 orig_stack_contexts = stack_context._state.contexts --> 326 yielded = next(result) 327 if stack_context._state.contexts is not orig_stack_contexts: 328 yielded = _create_future() /conda/envs/rapids/lib/python3.6/site-packages/dask_xgboost-0.1.5-py3.6.egg/dask_xgboost/core.py in _train(client, params, data, labels, dmatrix_kwargs, **kwargs) 135 label_parts = None 136 if isinstance(data, (list, tuple)): --> 137 if isinstance(data[0], Delayed): 138 for data_part in data: 139 if not isinstance(data_part, Delayed): IndexError: list index out of range

I'm running this on: VERSION_ID="7" PRETTY_NAME="CentOS Linux 7 (Core)" on a machine with four V100s Driver Version: 410.104 CUDA Version: 10.0

I ran the Jupyter notebook inside a docker container on the machine: docker run --runtime=nvidia --rm -it -p 9991:8888 -p 9992:8787 -p 9993:8786 -v /home/xxx/WORKSPACE/rapids-kml:/opt/gpudb/kml rapidsai/rapidsai:cuda10.0-runtime-ubuntu16.04

vilmara commented 5 years ago

Hi @starlett3 , did you solve the issue?