search

bowang-lab / scGPT

https://scgpt.readthedocs.io/en/latest/
MIT License
1.03k stars 203 forks source link

"Learning objective for pretraining" vs "Gene expression prediction." #222

Open noob000007 opened 4 months ago

noob000007 commented 4 months ago

image image

[ \mathbf{h}^{(i)}_n
] Isn't the shape of ( \mathbf{h} ) ( M \times D )? ( M ) represents the gene input. After passing through the MLP, it should be ( M \times 1 ), right? So how is the formula in the "Learning objective for pretraining" calculated? The dimensions don't match, do they?

noob000007 commented 4 months ago

image

subercui commented 3 months ago

Oh, do you mean it should be like equation 13 in the sense that there should be a subscript j? Yeah, I think that can be more accurate. Are you talking about the substraction in the first equation?

noob000007 commented 3 months ago

"First of all, thank you for your response. Secondly, I still have questions about the handling of 'unknown' and 'mask' in formulas 12 and 13 of the article. For 'unknown', does scGPT only input the embedding of gene tokens during processing, rather than the sum of embeddings for gene tokens and their corresponding expression tokens? For 'mask', is it inputting a special expression token called 'mask'?"

noob000007 commented 3 months ago

image "The selected positions with high prediction confidence" is no longer mentioned in the paper.

What does "high prediction confidence" mean, and how does scGPT determine which positions have high prediction confidence?