Cannot run Elmo model with multi-GPUs because of "ParameterList"

[bb-1357]

I use nn.DataParallel for running with multi-GPUs,However
"nn.ParameterList is being used with DataParallel but this is not supported. This list will appear empty for the models replicated on each GPU except the original one." the empty list really bothers me a lot，I wonder how to use the elmo model with multi-GPUs? nn.ParameterList appears in ELmo model just like this:

from allennlp.modules.elmo import Elmo
  (scalar_mix_0): ScalarMix(
    (scalar_parameters): ParameterList(
        (0): Parameter containing: [torch.FloatTensor of size 1]
        (1): Parameter containing: [torch.FloatTensor of size 1]
        (2): Parameter containing: [torch.FloatTensor of size 1]
    )
  )
  (scalar_mix_1): ScalarMix(
    (scalar_parameters): ParameterList(
        (0): Parameter containing: [torch.FloatTensor of size 1]
        (1): Parameter containing: [torch.FloatTensor of size 1]
        (2): Parameter containing: [torch.FloatTensor of size 1]
    )
  )
)

My allennlp Version is 1.3.0 and my pytorch version is 1.7.1. Thanks a lot. Hope you have a nice day !

[epwalsh]

Hi @bb-1357, it seems that the issues with ParameterList won't be fixed anytime soon. https://github.com/pytorch/pytorch/issues/36035#issuecomment-770960405

This is unfortunate, but as far as I can tell, the only place in our entire code base that uses this is the ScalarMix module. So it might not be too hard to make a fix for this one module that is backwards compatible.