iMoonLab / DeepHypergraph

A pytorch library for graph and hypergraph computation.
https://deephypergraph.com/
Apache License 2.0
671 stars 64 forks source link

您好在使用dhg.models.HGNN出现torch.sparse_coo_tensor的索引超出我的张量尺寸范围,具体的bug如下 #52

Closed JK-chance closed 8 months ago

JK-chance commented 8 months ago

Traceback (most recent call last): File "train.py", line 355, in main() File "train.py", line 195, in main train(opt, train_loader, model, epoch, val_loader) File "train.py", line 244, in train model.train_emb(images, captions, lengths,image_lengths=img_lengths) File "/home/iot/shared_pool_data3/shared_pool_data3/anaconda3/envs/CHANPY38/lib/python3.8/site-packages/torch/amp/autocast_mode.py", line 14, in decorate_autocast return func(*args, kwargs) File "/work/shared_pool_data3/Code/CHAN-main/lib/model.py", line 262, in train_emb img_emb, cap_emb = self.forward_emb(images, captions,lengths, image_lengths=image_lengths) File "/home/iot/shared_pool_data3/shared_pool_data3/anaconda3/envs/CHANPY38/lib/python3.8/site-packages/torch/amp/autocast_mode.py", line 14, in decorate_autocast return func(*args, *kwargs) File "/work/shared_pool_data3/Code/CHAN-main/lib/model.py", line 199, in forward_emb img_emb = self.img_hy(img_embhy,G) File "/home/iot/shared_pool_data3/shared_pool_data3/anaconda3/envs/CHANPY38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl return forward_call(input, kwargs) File "/home/iot/shared_pool_data3/shared_pool_data3/anaconda3/envs/CHANPY38/lib/python3.8/site-packages/dhg/models/hypergraphs/hgnn.py", line 44, in forward X = layer(X, hg) File "/home/iot/shared_pool_data3/shared_pool_data3/anaconda3/envs/CHANPY38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl return forward_call(*input, **kwargs) File "/home/iot/shared_pool_data3/shared_pool_data3/anaconda3/envs/CHANPY38/lib/python3.8/site-packages/dhg/nn/convs/hypergraphs/hgnn_conv.py", line 56, in forward X = hg.smoothing_with_HGNN(X).to(device) File "/home/iot/shared_pool_data3/shared_pool_data3/anaconda3/envs/CHANPY38/lib/python3.8/site-packages/dhg/structure/hypergraphs/hypergraph.py", line 1221, in smoothing_with_HGNN L_HGNN = self.L_HGNN File "/home/iot/shared_pool_data3/shared_pool_data3/anaconda3/envs/CHANPY38/lib/python3.8/site-packages/dhg/structure/hypergraphs/hypergraph.py", line 1180, in L_HGNN _tmp = self.D_v_neg_1_2.mm(self.H).mm(self.W_e).mm(self.D_e_neg_1).mm(self.H_T,).mm(self.D_v_neg_1_2) File "/home/iot/shared_pool_data3/shared_pool_data3/anaconda3/envs/CHANPY38/lib/python3.8/site-packages/dhg/structure/hypergraphs/hypergraph.py", line 929, in D_v_neg_1_2 _mat = self.D_v.clone() File "/home/iot/shared_pool_data3/shared_pool_data3/anaconda3/envs/CHANPY38/lib/python3.8/site-packages/dhg/structure/hypergraphs/hypergraph.py", line 863, in D_v _tmp = [self.D_v_of_group(name)._values().clone() for name in self.group_names] File "/home/iot/shared_pool_data3/shared_pool_data3/anaconda3/envs/CHANPY38/lib/python3.8/site-packages/dhg/structure/hypergraphs/hypergraph.py", line 863, in _tmp = [self.D_v_of_group(name)._values().clone() for name in self.group_names] File "/home/iot/shared_pool_data3/shared_pool_data3/anaconda3/envs/CHANPY38/lib/python3.8/site-packages/dhg/structure/hypergraphs/hypergraph.py", line 881, in D_v_of_group H = self.H_of_group(group_name).clone() File "/home/iot/shared_pool_data3/shared_pool_data3/anaconda3/envs/CHANPY38/lib/python3.8/site-packages/dhg/structure/hypergraphs/hypergraph.py", line 787, in H_of_group self.group_cache[group_name]["H"] = self.H_v2e_of_group(group_name) File "/home/iot/shared_pool_data3/shared_pool_data3/anaconda3/envs/CHANPY38/lib/python3.8/site-packages/dhg/structure/base.py", line 856, in H_v2e_of_group self.group_cache[group_name]["H_v2e"] = self._fetch_H_of_group("v2e", group_name) File "/home/iot/shared_pool_data3/shared_pool_data3/anaconda3/envs/CHANPY38/lib/python3.8/site-packages/dhg/structure/base.py", line 560, in _fetch_H_of_group H = torch.sparse_coo_tensor( RuntimeError: size is inconsistent with indices: for dim 0, size is 144 but found index 144

对源码的修改:HGNNconv forward改变了X的格式为float()以及更改了smoothingwithHGNN的设备,如下: class HGNNConv(nn.Module): r"""The HGNN convolution layer proposed in Hypergraph Neural Networks <https://arxiv.org/pdf/1809.09401>_ paper (AAAI 2019). Matrix Format:

.. math::
    \mathbf{X}^{\prime} = \sigma \left( \mathbf{D}_v^{-\frac{1}{2}} \mathbf{H} \mathbf{W}_e \mathbf{D}_e^{-1} 
    \mathbf{H}^\top \mathbf{D}_v^{-\frac{1}{2}} \mathbf{X} \mathbf{\Theta} \right).

where :math:`\mathbf{X}` is the input vertex feature matrix, :math:`\mathbf{H}` is the hypergraph incidence matrix, 
:math:`\mathbf{W}_e` is a diagonal hyperedge weight matrix, :math:`\mathbf{D}_v` is a diagonal vertex degree matrix, 
:math:`\mathbf{D}_e` is a diagonal hyperedge degree matrix, :math:`\mathbf{\Theta}` is the learnable parameters.

Args:
    ``in_channels`` (``int``): :math:`C_{in}` is the number of input channels.
    ``out_channels`` (int): :math:`C_{out}` is the number of output channels.
    ``bias`` (``bool``): If set to ``False``, the layer will not learn the bias parameter. Defaults to ``True``.
    ``use_bn`` (``bool``): If set to ``True``, the layer will use batch normalization. Defaults to ``False``.
    ``drop_rate`` (``float``): If set to a positive number, the layer will use dropout. Defaults to ``0.5``.
    ``is_last`` (``bool``): If set to ``True``, the layer will not apply the final activation and dropout functions. Defaults to ``False``.
"""

def __init__(
    self,
    in_channels: int,
    out_channels: int,
    bias: bool = True,
    use_bn: bool = False,
    drop_rate: float = 0.5,
    # drop_rate: float = 0,
    is_last: bool = False,
):
    super().__init__()
    self.is_last = is_last
    self.bn = nn.BatchNorm1d(out_channels) if use_bn else None
    self.act = nn.ReLU(inplace=True)
    self.drop = nn.Dropout(drop_rate)
    self.theta = nn.Linear(in_channels, out_channels, bias=bias)

def forward(self, X: torch.Tensor, hg: Hypergraph) -> torch.Tensor:
    r"""The forward function.

    Args:
        X (``torch.Tensor``): Input vertex feature matrix. Size :math:`(N, C_{in})`.
        hg (``dhg.Hypergraph``): The hypergraph structure that contains :math:`N` vertices.
    """
    device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
    X = self.theta(X)
    X=X.float()
    X = hg.smoothing_with_HGNN(X).to(device)
    if not self.is_last:
        X = self.act(X)
        if self.bn is not None:
            X = self.bn(X)

        X = self.drop(X)
    return X

请问大佬有没有办法解决这个问题

JK-chance commented 8 months ago

train only for one batchsize in HGNN can fix this problem but maybe produce another problem