search

xorbitsai / xorbits

Scalable Python DS & ML, in an API compatible & lightning fast way.
https://xorbits.readthedocs.io
Apache License 2.0
1.13k stars 69 forks source link

BUG:Encountered some issues with read_feather. #597

Closed smartpaopao closed 1 year ago

smartpaopao commented 1 year ago

Describe the bug

Encountered some issues with read_feather

To Reproduce

  1. Python version: python 3.8.9
  2. The version of Xorbits: xorbits 0.4.2
  3. numpy version :1.24.4 , pandas version :1.5.2
  4. Full stack of the error:

D:\python\lib\site-packages\xorbits\core\utils\fallback.py:35: RuntimeWarning: xorbits.pandas.read_feather will fallback to Pandas warnings.warn(warning_str, RuntimeWarning) D:\python\lib\site-packages\xorbits_mars\deploy\oscar\session.py:1873: UserWarning: No existing session found, creating a new local session now. warnings.warn(warning_msg) 2023-07-16 20:40:52,229 xorbits._mars.resource 20588 WARNING Unexpected error happened when gathering stats of device 0 Traceback (most recent call last): File "D:\python\lib\site-packages\xorbits_mars\resource.py", line 398, in cuda_card_stats device_status = nvutils.get_device_status(device_idx) File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 458, in get_device_status _nvml_check_error( File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 234, in _nvml_check_error raise NVMLAPIError(_error_str.decode(), errno=result) xorbits._mars.lib.nvutils.NVMLAPIError: (3) Not Supported 2023-07-16 20:40:52,340 xorbits._mars.resource 20588 WARNING Unexpected error happened when gathering stats of device 0 Traceback (most recent call last): File "D:\python\lib\site-packages\xorbits_mars\resource.py", line 398, in cuda_card_stats device_status = nvutils.get_device_status(device_idx) File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 458, in get_device_status _nvml_check_error( File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 234, in _nvml_check_error raise NVMLAPIError(_error_str.decode(), errno=result) xorbits._mars.lib.nvutils.NVMLAPIError: (3) Not Supported 2023-07-16 20:40:52,444 xorbits._mars.resource 20588 WARNING Unexpected error happened when gathering stats of device 0 Traceback (most recent call last): File "D:\python\lib\site-packages\xorbits_mars\resource.py", line 398, in cuda_card_stats device_status = nvutils.get_device_status(device_idx) File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 458, in get_device_status _nvml_check_error( File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 234, in _nvml_check_error raise NVMLAPIError(_error_str.decode(), errno=result) xorbits._mars.lib.nvutils.NVMLAPIError: (3) Not Supported 2023-07-16 20:40:52,498 xorbits._mars.resource 20588 WARNING Unexpected error happened when gathering stats of device 0 Traceback (most recent call last): File "D:\python\lib\site-packages\xorbits_mars\resource.py", line 398, in cuda_card_stats device_status = nvutils.get_device_status(device_idx) File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 458, in get_device_status _nvml_check_error( File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 234, in _nvml_check_error raise NVMLAPIError(_error_str.decode(), errno=result) xorbits._mars.lib.nvutils.NVMLAPIError: (3) Not Supported 2023-07-16 20:40:52,525 xorbits._mars.resource 20588 WARNING Unexpected error happened when gathering stats of device 0 Traceback (most recent call last): File "D:\python\lib\site-packages\xorbits_mars\resource.py", line 398, in cuda_card_stats device_status = nvutils.get_device_status(device_idx) File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 458, in get_device_status _nvml_check_error( File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 234, in _nvml_check_error raise NVMLAPIError(_error_str.decode(), errno=result) xorbits._mars.lib.nvutils.NVMLAPIError: (3) Not Supported 2023-07-16 20:40:52,749 xorbits._mars.resource 20588 WARNING Unexpected error happened when gathering stats of device 0 Traceback (most recent call last): File "D:\python\lib\site-packages\xorbits_mars\resource.py", line 398, in cuda_card_stats device_status = nvutils.get_device_status(device_idx) File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 458, in get_device_status _nvml_check_error( File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 234, in _nvml_check_error raise NVMLAPIError(_error_str.decode(), errno=result) xorbits._mars.lib.nvutils.NVMLAPIError: (3) Not Supported 2023-07-16 20:40:53,372 xorbits._mars.resource 20588 WARNING Unexpected error happened when gathering stats of device 0 Traceback (most recent call last): File "D:\python\lib\site-packages\xorbits_mars\resource.py", line 398, in cuda_card_stats device_status = nvutils.get_device_status(device_idx) File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 458, in get_device_status _nvml_check_error( File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 234, in _nvml_check_error raise NVMLAPIError(_error_str.decode(), errno=result) xorbits._mars.lib.nvutils.NVMLAPIError: (3) Not Supported 2023-07-16 20:40:53,538 xorbits._mars.resource 20588 WARNING Unexpected error happened when gathering stats of device 0 Traceback (most recent call last): File "D:\python\lib\site-packages\xorbits_mars\resource.py", line 398, in cuda_card_stats device_status = nvutils.get_device_status(device_idx) File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 458, in get_device_status _nvml_check_error( File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 234, in _nvml_check_error raise NVMLAPIError(_error_str.decode(), errno=result) xorbits._mars.lib.nvutils.NVMLAPIError: (3) Not Supported 2023-07-16 20:40:53,924 xorbits._mars.resource 20588 WARNING Unexpected error happened when gathering stats of device 0 Traceback (most recent call last): File "D:\python\lib\site-packages\xorbits_mars\resource.py", line 398, in cuda_card_stats device_status = nvutils.get_device_status(device_idx) File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 458, in get_device_status _nvml_check_error( File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 234, in _nvml_check_error raise NVMLAPIError(_error_str.decode(), errno=result) xorbits._mars.lib.nvutils.NVMLAPIError: (3) Not Supported 2023-07-16 20:40:53,925 xorbits._mars.deploy.oscar.local 20588 WARNING Web service started at http://127.0.0.1:54753 0%| | 0.00/100 [00:00<?, ?it/s]2023-07-16 20:40:54,401 xorbits._mars.resource 20588 WARNING Unexpected error happened when gathering stats of device 0 Traceback (most recent call last): File "D:\python\lib\site-packages\xorbits_mars\resource.py", line 398, in cuda_card_stats device_status = nvutils.get_device_status(device_idx) File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 458, in get_device_status _nvml_check_error( File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 234, in _nvml_check_error raise NVMLAPIError(_error_str.decode(), errno=result) xorbits._mars.lib.nvutils.NVMLAPIError: (3) Not Supported 2023-07-16 20:40:54,589 xorbits._mars.resource 20588 WARNING Unexpected error happened when gathering stats of device 0 Traceback (most recent call last): File "D:\python\lib\site-packages\xorbits_mars\resource.py", line 398, in cuda_card_stats device_status = nvutils.get_device_status(device_idx) File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 458, in get_device_status _nvml_check_error( File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 234, in _nvml_check_error raise NVMLAPIError(_error_str.decode(), errno=result) xorbits._mars.lib.nvutils.NVMLAPIError: (3) Not Supported 0%| | 0.00/100 [00:01<?, ?it/s]2023-07-16 20:40:55,425 xorbits._mars.resource 20588 WARNING Unexpected error happened when gathering stats of device 0 Traceback (most recent call last): File "D:\python\lib\site-packages\xorbits_mars\resource.py", line 398, in cuda_card_stats device_status = nvutils.get_device_status(device_idx) File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 458, in get_device_status _nvml_check_error( File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 234, in _nvml_check_error raise NVMLAPIError(_error_str.decode(), errno=result) xorbits._mars.lib.nvutils.NVMLAPIError: (3) Not Supported 2023-07-16 20:40:55,630 xorbits._mars.resource 20588 WARNING Unexpected error happened when gathering stats of device 0 Traceback (most recent call last): File "D:\python\lib\site-packages\xorbits_mars\resource.py", line 398, in cuda_card_stats device_status = nvutils.get_device_status(device_idx) File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 458, in get_device_status _nvml_check_error( File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 234, in _nvml_check_error raise NVMLAPIError(_error_str.decode(), errno=result) xorbits._mars.lib.nvutils.NVMLAPIError: (3) Not Supported 100%|██████████| 100.00/100 [00:02<00:00, 49.85it/s] Object 2023-07-16 20:40:56,466 xorbits._mars.resource 20588 WARNING Unexpected error happened when gathering stats of device 0 Traceback (most recent call last): File "D:\python\lib\site-packages\xorbits_mars\resource.py", line 398, in cuda_card_stats device_status = nvutils.get_device_status(device_idx) File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 458, in get_device_status _nvml_check_error( File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 234, in _nvml_check_error raise NVMLAPIError(_error_str.decode(), errno=result) xorbits._mars.lib.nvutils.NVMLAPIError: (3) Not Supported 2023-07-16 20:40:56,673 xorbits._mars.resource 20588 WARNING Unexpected error happened when gathering stats of device 0 Traceback (most recent call last): File "D:\python\lib\site-packages\xorbits_mars\resource.py", line 398, in cuda_card_stats device_status = nvutils.get_device_status(device_idx) File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 458, in get_device_status _nvml_check_error( File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 234, in _nvml_check_error raise NVMLAPIError(_error_str.decode(), errno=result) xorbits._mars.lib.nvutils.NVMLAPIError: (3) Not Supported 2023-07-16 20:40:57,498 xorbits._mars.resource 20588 WARNING Unexpected error happened when gathering stats of device 0 Traceback (most recent call last): File "D:\python\lib\site-packages\xorbits_mars\resource.py", line 398, in cuda_card_stats device_status = nvutils.get_device_status(device_idx) File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 458, in get_device_status _nvml_check_error( File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 234, in _nvml_check_error raise NVMLAPIError(_error_str.decode(), errno=result) xorbits._mars.lib.nvutils.NVMLAPIError: (3) Not Supported 2023-07-16 20:40:57,705 xorbits._mars.resource 20588 WARNING Unexpected error happened when gathering stats of device 0 Traceback (most recent call last): File "D:\python\lib\site-packages\xorbits_mars\resource.py", line 398, in cuda_card_stats device_status = nvutils.get_device_status(device_idx) File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 458, in get_device_status _nvml_check_error( File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 234, in _nvml_check_error raise NVMLAPIError(_error_str.decode(), errno=result) xorbits._mars.lib.nvutils.NVMLAPIError: (3) Not Supported 2023-07-16 20:40:58,513 xorbits._mars.resource 20588 WARNING Unexpected error happened when gathering stats of device 0 Traceback (most recent call last): File "D:\python\lib\site-packages\xorbits_mars\resource.py", line 398, in cuda_card_stats device_status = nvutils.get_device_status(device_idx) File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 458, in get_device_status _nvml_check_error( File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 234, in _nvml_check_error raise NVMLAPIError(_error_str.decode(), errno=result) xorbits._mars.lib.nvutils.NVMLAPIError: (3) Not Supported 2023-07-16 20:40:58,751 xorbits._mars.resource 20588 WARNING Unexpected error happened when gathering stats of device 0 Traceback (most recent call last): File "D:\python\lib\site-packages\xorbits_mars\resource.py", line 398, in cuda_card_stats device_status = nvutils.get_device_status(device_idx) File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 458, in get_device_status _nvml_check_error( File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 234, in _nvml_check_error raise NVMLAPIError(_error_str.decode(), errno=result) xorbits._mars.lib.nvutils.NVMLAPIError: (3) Not Supported 2023-07-16 20:40:59,544 xorbits._mars.resource 20588 WARNING Unexpected error happened when gathering stats of device 0 Traceback (most recent call last): File "D:\python\lib\site-packages\xorbits_mars\resource.py", line 398, in cuda_card_stats device_status = nvutils.get_device_status(device_idx) File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 458, in get_device_status _nvml_check_error( File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 234, in _nvml_check_error raise NVMLAPIError(_error_str.decode(), errno=result) xorbits._mars.lib.nvutils.NVMLAPIError: (3) Not Supported 2023-07-16 20:40:59,782 xorbits._mars.resource 20588 WARNING Unexpected error happened when gathering stats of device 0 Traceback (most recent call last): File "D:\python\lib\site-packages\xorbits_mars\resource.py", line 398, in cuda_card_stats device_status = nvutils.get_device_status(device_idx) File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 458, in get_device_status _nvml_check_error( File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 234, in _nvml_check_error raise NVMLAPIError(_error_str.decode(), errno=result) xorbits._mars.lib.nvutils.NVMLAPIError: (3) Not Supported 2023-07-16 20:41:00,573 xorbits._mars.resource 20588 WARNING Unexpected error happened when gathering stats of device 0 Traceback (most recent call last): File "D:\python\lib\site-packages\xorbits_mars\resource.py", line 398, in cuda_card_stats device_status = nvutils.get_device_status(device_idx) File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 458, in get_device_status _nvml_check_error( File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 234, in _nvml_check_error raise NVMLAPIError(_error_str.decode(), errno=result) xorbits._mars.lib.nvutils.NVMLAPIError: (3) Not Supported 2023-07-16 20:41:00,812 xorbits._mars.resource 20588 WARNING Unexpected error happened when gathering stats of device 0 Traceback (most recent call last): File "D:\python\lib\site-packages\xorbits_mars\resource.py", line 398, in cuda_card_stats device_status = nvutils.get_device_status(device_idx) File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 458, in get_device_status _nvml_check_error( File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 234, in _nvml_check_error raise NVMLAPIError(_error_str.decode(), errno=result) xorbits._mars.lib.nvutils.NVMLAPIError: (3) Not Supported 2023-07-16 20:41:01,616 xorbits._mars.resource 20588 WARNING Unexpected error happened when gathering stats of device 0 Traceback (most recent call last): File "D:\python\lib\site-packages\xorbits_mars\resource.py", line 398, in cuda_card_stats device_status = nvutils.get_device_status(device_idx) File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 458, in get_device_status _nvml_check_error( File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 234, in _nvml_check_error raise NVMLAPIError(_error_str.decode(), errno=result) xorbits._mars.lib.nvutils.NVMLAPIError: (3) Not Supported 2023-07-16 20:41:01,839 xorbits._mars.resource 20588 WARNING Unexpected error happened when gathering stats of device 0 Traceback (most recent call last): File "D:\python\lib\site-packages\xorbits_mars\resource.py", line 398, in cuda_card_stats device_status = nvutils.get_device_status(device_idx) File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 458, in get_device_status _nvml_check_error( File "D:\python\lib\site-packages\xorbits_mars\lib\nvutils.py", line 234, in _nvml_check_error raise NVMLAPIError(_error_str.decode(), errno=result) xorbits._mars.lib.nvutils.NVMLAPIError: (3) Not Supported

  1. Minimized code to reproduce the error. import xorbits.numpy as np import xorbits.pandas as pd

data_path = "./data/" close_1m = pd.read_feather(data_path + "close_1m.feather")

aresnow1 commented 1 year ago

Thanks for your report, we will fix this issue and suppress warnings.

aresnow1 commented 1 year ago

We have released new version and fixed this issue, you can install latest version via pip install -U xorbits and try again.