search

JackieHanLab / TOSICA

Transformer for One-Stop Interpretable Cell-type Annotation
MIT License
121 stars 23 forks source link

TOSICA pre function gives very different accuracy than evaluate in train function #19

Open Pinolinoo opened 10 months ago

Pinolinoo commented 10 months ago

Dear TOSICA makers,

thanks for this great tool I was very amazed by the paper.

I tried classifying my cells and got a very high evaluation accuracy during the training. However when I use the trained model for predictions the accuracy suddenly drops to very low level! Why does the evaluate function in the fit_train function give different results then the prediect function in the pre.py file?

best philipp

JiaweiChenGo commented 4 months ago

Thank you for your interest in TOSICA. I don't know why this is, did you load the pre-trained model (.pth) correctly?

IvyYang00 commented 1 month ago

Dear TOSICA makers,

thanks for this great tool I was very amazed by the paper.

I tried classifying my cells and got a very high evaluation accuracy during the training. However when I use the trained model for predictions the accuracy suddenly drops to very low level! Why does the evaluate function in the fit_train function give different results then the prediect function in the pre.py file?

best philipp

Hi! I encountered similar problem with you. And I solved it by loading other trained model. Specificly, the test accuracy is quite low when I used model-0.pth, and it can be improved significantly when using other model and the best accuracy is when using model-19.pth (my training epoch is 20).

Hope it helps!