Pre-public release wish list

kungfuai / kaishi

Tool kit to accelerate exploratory data analysis and data cleaning

https://kaishi.readthedocs.io/en/latest/

MIT License

11 stars 2 forks source link

Pre-public release wish list #25

Closed mwharton3 closed 4 years ago

mwharton3 commented 4 years ago

Filter by label (kaishi.core.filters)
To grayscale (kaishi.image.transforms)
Pixel average of all images (kaishi.image.aggregators)
Max dimension (kaishi.image.transforms)
Limit dimension (kaishi.image.transforms)
Standardize size (kaishi.image.transforms, could be combined with above transform)
With save, add functionality to save a kaishi directory in the root with a json of file report (incl labels), json of the pipeline (or yaml?), and any artifacts created (e.g. df_concatenated for tabular or pixel_average for image)
Train, validate, test split (kaishi.core.labelers)
Subsample (kaishi.core.filters)
Flatten (kaishi.core.restructurers? something like that?), flattens directory structure

mwharton3 commented 4 years ago

Tutorials

zzsi commented 4 years ago

CollapseChildren may not be needed for tabular data
TabularDataset currently does not print out data stats.

mwharton3 commented 4 years ago

I've been committing a lot of changes to more_pre_sxsw_feedback. For CollapseChildren, it's not run as part of a filter if it needs it (i.e. it will be run multiple times if multiple "duplicate" filter varieties are called). It's a very fast command to run though...

zzsi commented 4 years ago

should I pull in changes of more_pre_sxsw_feedback into zz_add_docs?

mwharton3 commented 4 years ago

That sounds good!

zzsi commented 4 years ago

[x] Add docstring to image related functions (@mwharton3 )
[x] Add MIT license
[x] Need to add to README: how to use tabular dataset (@zzsi )
[ ] upload to pypi (@zzsi )

mwharton3 commented 4 years ago

@zzsi there's a tutorial for tabular data in the docs now (basically the same flow as the image one).

I'll close this and make new issues for what's left over.