project-codeflare / codeflare

Simplifying the definition and execution, scaling and deployment of pipelines on the cloud.
https://codeflare.dev
Apache License 2.0
218 stars 35 forks source link

Adding a split utility to support data splitting -- enables key AIX36… #35

Closed raghukiran1224 closed 3 years ago

raghukiran1224 commented 3 years ago

AIX360 enabled usecase for parallelism;

Adding an example notebook to demo the capability in action with AIX360's SHAP explainer

Related Issue

Supports #33

Related PRs

N/A

This PR is not dependent on any other PR

What does this PR do?

This PR adds a split functionality that breaks the data into smaller chunks by user controlled splits. This enables scaling out of comptue intensive tasks using CodeFlare.

Description of Changes

Added a split function in utils along with a test. Some minor newlines etc for other files. Also a notebook to showcase the functionality of split in the context of AIX360 to accelerate explanations.

What gif most accurately describes how I feel towards this PR?

Example of a gif