xorbitsai / xorbits

Scalable Python DS & ML, in an API compatible & lightning fast way.
https://xorbits.readthedocs.io
Apache License 2.0
1.13k stars 69 forks source link

FEAT: Basic datasets feature #600

Closed codingl2k1 closed 1 year ago

codingl2k1 commented 1 year ago

What do these changes do?

Known issue:

Related issue number

Fixes #xxxx

Check code requirements

codecov[bot] commented 1 year ago

Codecov Report

Merging #600 (747a4ff) into main (d4092fa) will increase coverage by 0.10%. The diff coverage is 97.94%.

@@            Coverage Diff             @@
##             main     #600      +/-   ##
==========================================
+ Coverage   93.53%   93.64%   +0.10%     
==========================================
  Files        1012     1020       +8     
  Lines       78737    79029     +292     
  Branches    16358    16392      +34     
==========================================
+ Hits        73647    74006     +359     
+ Misses       3427     3363      -64     
+ Partials     1663     1660       -3     
Flag Coverage Δ
unittests 93.53% <97.60%> (+0.10%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
python/xorbits/core/data.py 99.11% <50.00%> (-0.89%) :arrow_down:
...n/xorbits/datasets/backends/huggingface/rechunk.py 93.75% <93.75%> (ø)
...bits/datasets/backends/huggingface/to_dataframe.py 96.77% <96.77%> (ø)
python/xorbits/datasets/dataset.py 97.56% <97.56%> (ø)
python/xorbits/_mars/core/entity/output_types.py 84.48% <100.00%> (+0.27%) :arrow_up:
python/xorbits/_mars/utils.py 89.64% <100.00%> (+0.04%) :arrow_up:
python/xorbits/datasets/adapter.py 100.00% <100.00%> (ø)
...thon/xorbits/datasets/backends/huggingface/core.py 100.00% <100.00%> (ø)
...on/xorbits/datasets/backends/huggingface/loader.py 100.00% <100.00%> (ø)
...ython/xorbits/datasets/backends/huggingface/map.py 100.00% <100.00%> (ø)
... and 1 more

... and 13 files with indirect coverage changes