Trouble reproducing Genomics Benchmark Result

HazyResearch / hyena-dna

Official implementation for HyenaDNA, a long-range genomic foundation model built with Hyena

https://arxiv.org/abs/2306.15794

Apache License 2.0

532 stars 74 forks source link

Trouble reproducing Genomics Benchmark Result #37

Closed yair-schiff closed 6 months ago

yair-schiff commented 6 months ago

Hello,

Thank you for the great work and repo!

I am trying to reproduce the HyenaDNA column of Table 4.1 (GenomicsBenchmark). I am using the weights from LongSafari/hyenadna-tiny-1k-seqlen. However, I am unable to reproduce the results.

Would it be possible for you to specify which hyperparms from Table A.3 (screenshot below) were used for each of the datasets in this benchmark:

exnx commented 6 months ago

Sorry we don't have those readily available.

yair-schiff commented 6 months ago

Thanks for the reply!