xorbitsai / xorbits

Scalable Python DS & ML, in an API compatible & lightning fast way.
https://xorbits.readthedocs.io
Apache License 2.0
1.1k stars 67 forks source link

BUG: IPython PlainTextFormatter raises TypeError: object of type '_Largest' has no len() #650

Open codingl2k1 opened 1 year ago

codingl2k1 commented 1 year ago

Describe the bug

A clear and concise description of what the bug is.

Jupyter Notebook pretty print a arrow dtype dataframe errror:

TypeError                                 Traceback (most recent call last)
File ~/miniconda3/lib/python3.9/site-packages/IPython/core/formatters.py:706, in PlainTextFormatter.__call__(self, obj)
    699 stream = StringIO()
    700 printer = pretty.RepresentationPrinter(stream, self.verbose,
    701     self.max_width, self.newline,
    702     max_seq_length=self.max_seq_length,
    703     singleton_pprinters=self.singleton_printers,
    704     type_pprinters=self.type_printers,
    705     deferred_pprinters=self.deferred_printers)
--> 706 printer.pretty(obj)
    707 printer.flush()
    708 return stream.getvalue()

File ~/miniconda3/lib/python3.9/site-packages/IPython/lib/pretty.py:410, in RepresentationPrinter.pretty(self, obj)
    407                         return meth(obj, self, cycle)
    408                 if cls is not object \
    409                         and callable(cls.__dict__.get('__repr__')):
--> 410                     return _repr_pprint(obj, self, cycle)
    412     return _default_pprint(obj, self, cycle)
    413 finally:

File ~/miniconda3/lib/python3.9/site-packages/IPython/lib/pretty.py:778, in _repr_pprint(obj, p, cycle)
    776 """A pprint that just redirects to the normal repr function."""
    777 # Find newlines and replace them with p.break_()
--> 778 output = repr(obj)
    779 lines = output.splitlines()
    780 with p.group():

File ~/xorbits/python/xorbits/utils.py:38, in safe_repr_str.<locals>.inn(self, *args, **kwargs)
     36     return getattr(object, f.__name__)(self)
     37 else:
---> 38     return f(self, *args, **kwargs)

File ~/xorbits/python/xorbits/core/data.py:310, in DataRef.__repr__(self)
    308     return self.data._mars_entity.op.data.__repr__()
    309 else:
--> 310     run(self)
    311     return self.data.__repr__()
File /work/python/xorbits/_mars/services/task/supervisor/processor.py:388, in run()

File /work/python/xorbits/_mars/services/task/supervisor/processor.py:265, in _process_stage_chunk_graph()

File /work/python/xorbits/_mars/services/task/execution/mars/executor.py:203, in execute_subtask_graph()

File /work/python/xorbits/_mars/services/task/execution/mars/stage.py:233, in run()

File /work/python/xorbits/_mars/services/task/execution/mars/stage.py:253, in _run()

File /work/python/xorbits/_mars/services/subtask/worker/processor.py:212, in _execute_operand()

File /work/python/xorbits/_mars/core/operand/core.py:492, in execute()

File /work/python/xorbits/_mars/dataframe/sort/psrs.py:792, in execute()

File /work/python/xorbits/_mars/dataframe/sort/psrs.py:747, in _execute_map()

File /work/python/xorbits/_mars/dataframe/sort/psrs.py:622, in _execute_dataframe_map()

File /opt/conda/lib/python3.9/site-packages/pandas/util/_decorators.py:331, in wrapper()

File /opt/conda/lib/python3.9/site-packages/pandas/core/frame.py:5635, in fillna()

File /opt/conda/lib/python3.9/site-packages/pandas/core/generic.py:6922, in fillna()

File /opt/conda/lib/python3.9/site-packages/pandas/core/internals/managers.py:443, in fillna()

File /opt/conda/lib/python3.9/site-packages/pandas/core/internals/managers.py:352, in apply()

File /opt/conda/lib/python3.9/site-packages/pandas/core/internals/blocks.py:1586, in fillna()

File /opt/conda/lib/python3.9/site-packages/pandas/core/internals/blocks.py:1200, in fillna()

File /opt/conda/lib/python3.9/site-packages/pandas/core/internals/blocks.py:1457, in where()

File /opt/conda/lib/python3.9/site-packages/pandas/core/arrays/base.py:1547, in _where()

File /opt/conda/lib/python3.9/site-packages/pandas/core/arrays/arrow/array.py:898, in __setitem__()

TypeError: object of type '_Largest' has no len()

To Reproduce

To help us to reproduce this bug, please provide information below:

  1. Your Python version
  2. The version of Xorbits you use
  3. Versions of crucial packages, such as numpy, scipy and pandas
  4. Full stack of the error.
  5. Minimized code to reproduce the error.

Expected behavior

A clear and concise description of what you expected to happen.

Additional context

Add any other context about the problem here.