Currently, DataSetProviders cannot control multiprocessing of given Featurizers.
multiprocessing is controlled at the Featurizer.featurize() level
need to change DataSetProvider.featurize() method to pass multiprocessing parameters to each Featurizer.featurize() call
usually, DataSetProviders also call PipelineFeaturizer and ConcatenatedFeaturizer, those need to be able to switch between multiprocessing and not-multiprocessing, depending on if the specific Featurizer inherits from BaseFeaturizer or ParallelBaseFeaturizer
schallerdavid
commented
3 years ago
Currently, DataSetProviders cannot control multiprocessing of given Featurizers.