CaibinSh
commented
2 years ago Hi @KalinNonchev , thanks for reaching out.
Both epoch number and batch size are important for training. The model is updated after each batch. Therefore, the total number of updates is equal to (epoch * sample_size / batch_size) -- that might explain why it is sensitive to batch size.
If the memory is enough, you can increase both batch size and epoch. In the case of 1000 batch size, you can go up to 1000 epochs or even more.
We used 800 epochs + 64 batch sizes to ensure the best performance. This combination is definitely enough for nearly all the cases, but it is expensive, in terms of computation. (~150 epochs + 64 batch size generated similar results). So in the tutorials, we take 400 epochs to save time.
Yes, we use a small batch size due to the computational limitations -- the gpu memory is small (4G) in one of our devices. A big batch size created errors.
Please do not hesitate to contact me if there are further questions. Best, Caibin
Dear scAR-Team,
thank you for developing this package. I am currently exploring it and I would like to ask you
1) how do you determine the number of epochs the user should use for feature_type = "mRNA"? In your tutorials you used 400 epochs and in your paper you mentioned that you fixed the epochs to 800. I applied it for various batch sizes (up to 1000) and noticed that the model is sensitive to it.
2) I noticed that you use rather small batch-size - is scAR sensitive to the batch-size, it is just due to computational limitations or due to better perfromance?
Thank you in advance!
Best,