lucidrains / enformer-pytorch

Implementation of Enformer, Deepmind's attention network for predicting gene expression, in Pytorch
MIT License
435 stars 82 forks source link

The pytorch-enformer has much lower performance than the tensorflow-enformer #44

Closed biginfor closed 1 month ago

biginfor commented 4 months ago

Dear developers, thank you very much for developing the pytorch version of enformer! However, when I use the example data provided by basenji and my own ATAC-Seq dataset, the predictive power of pytorch-enformer compared to the predictive power of tensorflow-enformer (referring to the Pearson correlation coefficient) drops by around 0.2, (0.3 vs 0.5 for the validation set) . I tried to initialize my model weights following the weight initialization method of the tensorflow-enformer, but things didn't get radically better, can anyone tell me why?

Best, Eli