search

fteufel / SecretoGen

A conditional generative model for signal peptide design and efficiency prediction
https://openreview.net/forum?id=vXXEfmYsvS
BSD 3-Clause "New" or "Revised" License
4 stars 2 forks source link

Evaluated perplexity not equal to generated perplexity #3

Open tony9664 opened 1 month ago

tony9664 commented 1 month ago

When I generate new SP sequences, a perplexity value will be provided for each sequence in the CSV file. However, when I tried to re-calculate the perplexity values for the new sequences using the run_perplexity.py script, I noticed that the re-calculated values are generally higher than the ones provided when generating the sequences. Is this an expected behavior? Since the paper considered the perplexity as an indicator of SP efficiency, which value should I trust more?

fteufel commented 1 month ago

Will have to check. The perplexity from the scoring script should be reliable, as this is the one that was used in the paper.