search

JJGO / hyperlight

Modular and intuitive Hypernetworks in Pytorch
Apache License 2.0
32 stars 3 forks source link

About batch processing for main architecture RNN #4

Open murataryusei opened 1 year ago

murataryusei commented 1 year ago

Hello, Thank you for open-sourcing this repository! If the main architecture is RNN, how should I implement batch processing?

JJGO commented 1 year ago

Could you elaborate about what do you mean by batch processing? By default, in hyperlight, a batch of data uses the same hypernetwork input and thus the same weights. To use multiple sets of weights within the same batch, the recommended way to do it is using gradient accumulation.

murataryusei commented 1 year ago

Thank you for your reply! Ultimately, I have resolved my question regarding batch processing.