Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.
Apache License 2.0
1.76k
stars
281
forks
source link
Can we input a custom collate function as an input variable when creating the dataloader ? #759
Similar to the transform_spec function, is there a way to input a custom collator function?