Traceback (most recent call last):
File "/home/rainbow/miniconda3/envs/test_spark/lib/python3.7/site-packages/xgboost_ray/main.py", line 1181, in _train
ray.get(ready)
File "/home/rainbow/miniconda3/envs/test_spark/lib/python3.7/site-packages/ray/_private/client_mode_hook.py", line 105, in wrapper
return func(*args, **kwargs)
File "/home/rainbow/miniconda3/envs/test_spark/lib/python3.7/site-packages/ray/_private/worker.py", line 2309, in get
PC: @ 0x7f9ddb1e3c90 (unknown) accept4
@ 0x7f9ddb42c3c0 (unknown) (unknown)
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(TypeError): ray::_RemoteRayXGBoostActor.train() (pid=1649778, ip=10.239.158.60, repr=<xgboost_ray.main._RemoteRayXGBoostActor object at 0x7f8fe6c50dd0>)
File "/home/rainbow/miniconda3/envs/test_spark/lib/python3.7/site-packages/xgboost_ray/main.py", line 630, in train
self.load_data(dtrain)
File "/home/rainbow/miniconda3/envs/test_spark/lib/python3.7/site-packages/xgboost_ray/main.py", line 596, in load_data
param = data.get_data(self.rank, self.num_actors)
File "/home/rainbow/miniconda3/envs/test_spark/lib/python3.7/site-packages/xgboost_ray/matrix.py", line 847, in get_data
self.load_data(num_actors=num_actors, rank=rank)
File "/home/rainbow/miniconda3/envs/test_spark/lib/python3.7/site-packages/xgboost_ray/matrix.py", line 834, in load_data
self.num_actors, self.sharding, rank=rank)
File "/home/rainbow/miniconda3/envs/test_spark/lib/python3.7/site-packages/xgboost_ray/matrix.py", line 558, in load_data
**self.kwargs)
File "/home/rainbow/miniconda3/envs/test_spark/lib/python3.7/site-packages/xgboost_ray/data_sources/ray_dataset.py", line 60, in load_data
local_df = [ds.to_pandas(limit=DATASET_TO_PANDAS_LIMIT) for ds in data]
File "/home/rainbow/miniconda3/envs/test_spark/lib/python3.7/site-packages/xgboost_ray/data_sources/ray_dataset.py", line 60, in <listcomp>
local_df = [ds.to_pandas(limit=DATASET_TO_PANDAS_LIMIT) for ds in data]
File "/home/rainbow/miniconda3/envs/test_spark/lib/python3.7/site-packages/ray/data/dataset.py", line 3391, in to_pandas
output.add_block(ray.get(block))
File "/home/rainbow/miniconda3/envs/test_spark/lib/python3.7/site-packages/ray/data/_internal/delegating_block_builder.py", line 56, in add_block
self._builder.add_block(block)
File "/home/rainbow/miniconda3/envs/test_spark/lib/python3.7/site-packages/ray/data/_internal/table_block.py", line 74, in add_block
f"Got a block of type {type(block)}, expected {self._block_type}."
TypeError: Got a block of type <class 'bytes'>, expected <class 'pyarrow.lib.Table'>.If you are mapping a function, ensure it returns an object with the expected type. Block:
b'\xff\xff\xff\xff\xd8\x05\x00\x00\x10\x00\x00\x00\x00\x00\n\x00\x0e\x00\x06\x00\r\x00\x08\x00\n\x00\x00\x00\x00\x00\x04\x00\x10\x00\x00\x00\x00\x01\n\x00\x0c\x00\x0...