Dataset/dataloader API

[vvmnnnkv]

Feature Description

Add helper class that to make easier data loading, batching, shuffling, etc. I.e. dataset and/or dataloader class API that would help user to prepare data for training.

Use cases:

• Easify and standardize data preparation on the client side, across all libraries.
• Supply dataset into Job.train(..., {data: dataset, ...}) and update Job.train to work with the dataset API.

Design: TBD

Is your feature request related to a problem?

Standardize the way data is loaded into training loop across workers.

What alternatives have you considered?

Discussed with FL team.

Additional Context

See also #264

[pengyuan-zhou]

On it

[pengyuan-zhou]

268

[vkkhare]

291 closes this