search

ContinualAI / avalanche

Avalanche: an End-to-End Library for Continual Learning based on PyTorch.
http://avalanche.continualai.org
MIT License
1.76k stars 287 forks source link

PNN adapter arguement missing in the _init_() method #1625

Open sudaksh14 opened 6 months ago

sudaksh14 commented 6 months ago

There is some issue with the class LinearAdapter for PNN models in PNN.py The variable self.num_prev_modules is not initialized anywhere, see below code snippet.

sudaksh14 commented 6 months ago

The below is the src code for class LinearAdapter in PNN.py module:


class LinearAdapter(nn.Module):
    """
    Linear adapter for Progressive Neural Networks.
    """

    def __init__(self, in_features, out_features_per_column, num_prev_modules):
        """
        :param in_features: size of each input sample
        :param out_features_per_column: size of each output sample
        :param num_prev_modules: number of previous modules
        """
        super().__init__()
        # Eq. 1 - lateral connections
        # one layer for each previous column. Empty for the first task.
        self.lat_layers = nn.ModuleList([])
        for _ in range(num_prev_modules):
            m = nn.Linear(in_features, out_features_per_column)
            self.lat_layers.append(m)

    def forward(self, x):
        assert len(x) == self.num_prev_modules
        hs = []
        for ii, lat in enumerate(self.lat_layers):
            hs.append(lat(x[ii]))
        return sum(hs)

I get an error when I select my adapter="linear" as follow:

0it [00:00, ?it/s]Traceback (most recent call last):
  File "/home/aku7rng/git/generalized_timeseries_processing/internal_backbone/SOTA.py", line 389, in <module>
    strategy.train(experience)
  File "/home/aku7rng/.conda/envs/sud_env/lib/python3.9/site-packages/avalanche/training/templates/base_sgd.py", line 211, in train
    super().train(experiences, eval_streams, **kwargs)
  File "/home/aku7rng/.conda/envs/sud_env/lib/python3.9/site-packages/avalanche/training/templates/base.py", line 163, in train
    self._train_exp(self.experience, eval_streams, **kwargs)
  File "/home/aku7rng/.conda/envs/sud_env/lib/python3.9/site-packages/avalanche/training/templates/base_sgd.py", line 337, in _train_exp
    self.training_epoch(**kwargs)
  File "/home/aku7rng/.conda/envs/sud_env/lib/python3.9/site-packages/avalanche/training/templates/update_type/sgd_update.py", line 31, in training_epoch
    self.mb_output = self.forward()
  File "/home/aku7rng/.conda/envs/sud_env/lib/python3.9/site-packages/avalanche/training/templates/problem_type/supervised_problem.py", line 47, in forward
    return avalanche_forward(self.model, self.mb_x, self.mb_task_id)
  File "/home/aku7rng/.conda/envs/sud_env/lib/python3.9/site-packages/avalanche/models/utils.py", line 20, in avalanche_forward
    return model(x, task_labels)
  File "/home/aku7rng/.conda/envs/sud_env/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aku7rng/.conda/envs/sud_env/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aku7rng/.conda/envs/sud_env/lib/python3.9/site-packages/avalanche/models/dynamic_modules.py", line 163, in forward
    out_task = self.forward_single_task(x_task, task.item())
  File "/home/aku7rng/.conda/envs/sud_env/lib/python3.9/site-packages/avalanche/models/pnn.py", line 276, in forward_single_task
    x = [F.relu(el) for el in lay(x, task_label)]
  File "/home/aku7rng/.conda/envs/sud_env/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aku7rng/.conda/envs/sud_env/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aku7rng/.conda/envs/sud_env/lib/python3.9/site-packages/avalanche/models/dynamic_modules.py", line 155, in forward
    return self.forward_single_task(x, task_labels)
  File "/home/aku7rng/.conda/envs/sud_env/lib/python3.9/site-packages/avalanche/models/pnn.py", line 217, in forward_single_task
    hs.append(self.columns[ii](x[: ii + 1]))
  File "/home/aku7rng/.conda/envs/sud_env/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aku7rng/.conda/envs/sud_env/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aku7rng/.conda/envs/sud_env/lib/python3.9/site-packages/avalanche/models/pnn.py", line 126, in forward
    hs = self.adapter(prev_xs)
  File "/home/aku7rng/.conda/envs/sud_env/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aku7rng/.conda/envs/sud_env/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aku7rng/.conda/envs/sud_env/lib/python3.9/site-packages/avalanche/models/pnn.py", line 31, in forward
    assert len(x) == self.num_prev_modules
  File "/home/aku7rng/.conda/envs/sud_env/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1688, in __getattr__
    raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'")
AttributeError: 'LinearAdapter' object has no attribute 'num_prev_modules'
  0%|

This is because the init method does not initialize the variable self.num_prev_modules, could you please check this.

Thanks!