[suggestion] batch inference module - merge sections to better present Ray AIR - Githubissues

ray-project / ray-educational-materials

This is suite of the hands-on training materials that shows how to scale CV, NLP, time-series forecasting workloads with Ray.

Apache License 2.0

344 stars 65 forks source link

[suggestion] batch inference module - merge sections to better present Ray AIR #38

Closed kamil-kaczmarek closed 1 year ago

kamil-kaczmarek commented 1 year ago

Please share your suggestion here

Merge Datasets and BatchPredictor approaches into one: "Distributed batch inference with Ray AIR".

Datasets approach is more basic; BatchPredictor is more specialized, easy to use and feature rich as it also:

supports various predictos (TorchPredictor, HFPredictor)
handles framework native batch conversions
give an options to resume operations from AIR checkpoint to prediction, selection / keep columns, etc.

Note in this section that BatchPredictor calls dataset.map_batches() under the hood. From that perspective they are similar.

kamil-kaczmarek commented 1 year ago

implemented in #52