does the training script support DataParallel training?

[jackie930]

hi,

i am trying to reproduce the result on a fairly large dataset, and i tried below codes on a 8-gpu instance, it seems current repo doesn't support parallel training, just post this to see if my understanding correct.

#!pip install -U pyabsa
from pyabsa.functional import ATEPCModelList
from pyabsa.functional import Trainer, ATEPCTrainer
from pyabsa.functional import ABSADatasetList
from pyabsa.functional import ATEPCConfigManager

atepc_config_custom = ATEPCConfigManager.get_atepc_config_chinese()
atepc_config_custom.num_epoch = 5
atepc_config_custom.evaluate_begin = 1
atepc_config_custom.log_step = 100
atepc_config_custom.model = ATEPCModelList.LCF_ATEPC

aspect_extractor = ATEPCTrainer(config=atepc_config_custom, 
                                dataset='./custom_1109'
                                )

[yangheng95]

Yes, the DataParallel is not considered before, as ABSA usually has limited data. However, this feature should be enabled by several lines' modifications, such as codes of dataloader and model mapping. Temporarily, I am unavailable to add this feature. But if you did this, I would like you to PR and it will help others having the same demand.

[jackie930]

Yes, the DataParallel is not considered before, as ABSA usually has limited data. However, this feature should be enabled by several lines' modifications, such as codes of dataloader and model mapping. Temporarily, I am unavailable to add this feature. But if you did this, I would like you to PR and it will help others having the same demand.

great! i will try to work on this one then!

[jackie930]

close the issue as it's fixed