xorbitsai / xorbits

Scalable Python DS & ML, in an API compatible & lightning fast way.
https://xorbits.readthedocs.io
Apache License 2.0
1.11k stars 67 forks source link

BUG: TypeError: data type 'string' not understood #635

Open codingl2k1 opened 1 year ago

codingl2k1 commented 1 year ago

Describe the bug

A clear and concise description of what the bug is.

To Reproduce

To help us to reproduce this bug, please provide information below:

import xorbits.pandas as pd
pdf = pd.DataFrame({"A":[1, 2, 3], "B": [["1"], ["2"], ["3"]]})
pdf["B"].map(lambda x: x[0], dtype="string")
Traceback (most recent call last):
  File "/Users/codingl2k1/Work/xorbits/python/xorbits/pandas/mars_adapters/core.py", line 179, in wrapped
    return from_mars(c(*new_args, **new_kwargs))
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/codingl2k1/Work/xorbits/python/xorbits/_mars/dataframe/base/map.py", line 275, in series_map
    return op(series, dtype=dtype, skip_infer=skip_infer)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/codingl2k1/Work/xorbits/python/xorbits/_mars/core/mode.py", line 78, in _inner
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/codingl2k1/Work/xorbits/python/xorbits/_mars/dataframe/base/map.py", line 112, in __call__
    dtype = np.dtype(dtype)
            ^^^^^^^^^^^^^^^
TypeError: data type 'string' not understood

xorbits.pandas should support pd dtypes: https://pandas.pydata.org/docs/user_guide/text.html

  1. Your Python version
  2. The version of Xorbits you use
  3. Versions of crucial packages, such as numpy, scipy and pandas
  4. Full stack of the error.
  5. Minimized code to reproduce the error.

Expected behavior

A clear and concise description of what you expected to happen.

Additional context

Add any other context about the problem here.