Closed Jch520 closed 2 years ago
Could you provide your config file?
I use the config file the github provide.I only do the connectio abalation
[ // residual connection { name: 'residual-conn', parents: [ 'default', 'data/snli', ], connection: 'residual', repeat: 4, }, { name: 'residual-conn', parents: [ 'default', 'data/scitail', ], repeat: 4, connection: 'residual' }, { name: 'residual-conn', parents: [ 'default', 'data/quora', ], repeat: 4, connection: 'residual' }, ]
OK, I'll look into it later.
@Jch520 fix by 4ce190e88f5dbc19f16fb69229e641e5ac50aa73
Background: The original experiments are run in Tensorflow. This pytorch implementation is a re-implementation that was created later. I checked the "default" experiment results and some of the ablations are the same as the Tensorflow version, but not all due to the time limit. In Tensorflow the dimensions are auto-assigned but in PyTorch we'll have to compute the exact numbers for all weights, so I overlooked this difference when first implementing it.
Hi,when I did the abalation experiment, an error occurred in the residual connection module Traceback (most recent call last): File "train.py", line 49, in
main()
File "train.py", line 32, in main
states = trainer.train()
File "/hy-tmp/res_ave/src/trainer.py", line 52, in train
stats = model.update(batch)
File "/hy-tmp/res_ave/src/model.py", line 69, in update
output = self.network(inputs)
File "/usr/local/miniconda3/envs/res_torch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, kwargs)
File "/hy-tmp/res_ave/src/network.py", line 59, in forward
a_enc = block['encoder'](a, mask_a)
File "/usr/local/miniconda3/envs/res_torch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, *kwargs)
File "/hy-tmp/res_ave/src/modules/encoder.py", line 38, in forward
x = encoder(x)
File "/usr/local/miniconda3/envs/res_torch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(input, kwargs)
File "/hy-tmp/res_ave/src/modules/init.py", line 105, in forward
return torch.cat([encoder(x) for encoder in self.model], dim=-1)
File "/hy-tmp/res_ave/src/modules/init.py", line 105, in
return torch.cat([encoder(x) for encoder in self.model], dim=-1)
File "/usr/local/miniconda3/envs/res_torch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, *kwargs)
File "/usr/local/miniconda3/envs/res_torch/lib/python3.7/site-packages/torch/nn/modules/container.py", line 92, in forward
input = module(input)
File "/usr/local/miniconda3/envs/res_torch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(input, **kwargs)
File "/usr/local/miniconda3/envs/res_torch/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 187, in forward
self.padding, self.dilation, self.groups)
RuntimeError: Given groups=1, weight of size [150, 450, 3], expected input[256, 150, 38] to have 450 channels, but got 150 channels instead.
Thank you