Closed hanhongxiaozhanggui closed 2 months ago
Hi @hanhongxiaozhanggui,
Nope, the base class method and the override methods are still there and are used to create new processes as a part of the multi_process_manager()
function. Note that function, preprocess_dataset_subprocess()
is implemented for each custom dataloader, and can be implemented for your own custom dataloader if you wish.
Are you having trouble implementing your own custom dataloader? If not, did you run into some issue where the existing method implementations seem to be missing?
@hanhongxiaozhanggui, closing this for the time being since there doesn't seem to be any additional discussion, but do feel free to re-open or create a new issue if any confusion on this persists.
I find a class atribute which named "self.preprocess_dataset_subprocess" is used in the function "multi_process_manager" of Class Baseloader. Does the atribute has been drop in the recentest version of your project?