automl / RNAformer

Scalable Deep Learning for RNA Secondary Structure Prediction
Apache License 2.0
28 stars 9 forks source link

Unable to reproduce results on bpRNA #8

Open GalAvineri opened 6 months ago

GalAvineri commented 6 months ago

I attempted to reproduce your results on the bpRNA dataset by following the instructions in the README file. However I got results that are different than the published ones.

My Results

solved accuracy precision recall f1_score mcc num_samples test_set
0.00229885 0.000382429 0.304632 0.0637105 0.0917703 0.121859 1305 bprna_ts0

Expected Results (from the paper)

solved f1_score mcc
17.2% 0.728 0.733

Perhaps I did something wrong during the setup?

JoergFranke commented 5 months ago

This should be fixed with #7