How to create mini batch and handle laplacian matrix when training?

thunlp / OpenNE

An Open-Source Package for Network Embedding (NE)

MIT License

1.68k stars 485 forks source link

How to create mini batch and handle laplacian matrix when training? #104

Closed hxtruong6 closed 3 years ago

hxtruong6 commented 4 years ago

Thanks for your SDNE code algorithm. I also re-implementation code but I stuck with create mini batch for each epoch. I don't know how to create mini batch. I suppose a mini batch is a adjacency matrix with length i->j: A[i->j, :].

And I don't see your code use laplacian matrix when training like the original paper? Loss of 1st should be 2tr(Y.T * L * Y)

Thanks your reading and helping.

Bznkxs commented 4 years ago

Hi @hxtruong, You can create a mini batch of size bs by choosing bs random indices and using indexing notation:

index = torch.randint(high=self.node_size, size=[self.bs])
adj_batch_train = self.adj_mat[index, :]
adj_mat_train = adj_batch_train[:, index]

You can refer to src/openne/models/sdne.py in OpenNE-PyTorch for details.

Also, regarding your second problem, you can find that our implementation of loss_1st is actually equivalent to 2*tr(Y.T * L * Y), so no problem there.