Notebooks:
extract.ipynb
-
eda.ipynb
-
aa_predict.ipynb
-
aa_predict_clust.ipynb
-
cluster.ipynb
-
coord_predict.ipynb
-
Folders:
wisp
- folder with code for model training.
data
- folder with datasets.
loss
- visualisation of training losses.
pred
- visualisation of predicted coordinates.
plots
- various plots.
Data is highly variable due to different structures and stochasticity in the optimization procedure (length - 11, X and Y coordinates, distribution of differences among the max and min values per position):
There are no critical differences for CDR sequences, therefore for each sequence we chose the mean coordinates. However, canonical and putative sequences are much more diverse and more sophisticated procedure is needed. For each unique sequence we choose each other kmer with differences in MSE and minimal MSE (per sequence) lower than 0.005:
Then we choose the mean coordinate values (like in CDRs) per sequence.
Numbers of the sequences / unique sequences per length per data type:
CDR
L all uniq
10 1 1
11 21 14
12 32 16
13 69 37
14 46 31
15 42 25
16 32 11
Canonical
L all uniq
10 176 60
11 910 171
12 452 97
13 395 84
14 167 24
15 109 17
16 80 4
Putative
L all uniq
10 7219 1648
11 179175 23598
12 146165 17408
13 74735 10167
14 7177 1463
15 7357 1254
16 2697 481
Surround sequences by null symbols.
First and last amino acids depends on each other other. It is better than the naive approach:
Add position or position/length for each amino acid to each dense layer. It helps:
Interstinegly enough, if we train the model on CDRs and predict putative ones than the pos-len modification helps a lot:
--
window - sliding window, predict the coordinate in the centre of the window.
directional / bidirectional - recurrent neural networks.
straight - predict each coordinate.
difference - predict differences between neighbour coordinates
window + straight - predict each coordinate.
Surround sequences by null symbols.
First and last amino acids depends on each other other. It is better than the naive approach:
Add position or position/length for each amino acid to each dense layer. It helps:
Interstinegly enough, if we train the model on CDRs and predict putative ones than the pos-len modification helps a lot:
GRU models, 1500 iterations:
GRU models, 1500 iterations:
Due to the iterative nature the error is aggreageted through the sequence.
Ensemble the predicted coordinates with Bayesian Ridge
. Cross validation score (negative MSE):
array([-0.0673693 , -0.1109571 , -0.39486038, -0.09391625, -0.02850241,
-0.13117736, -0.10696883, -0.18005811, -0.13272101, -0.20850138])
one-hot - code each amino acid as a 20-dimensional vector with 1 at the index of the corresponding amino acid.
kidera - worked worse than one-hot.
embeddings - ??? (infer embeddings on the overall dataю)
two-hot - one-hot vector + one-hot vector for target amino acid.
Scale the data to [0,1]. Tested on models with [128,64] dense layers, 4-4 window size, 2000 iterations.
MSE:
no scale:
cdr - 1.61775 ***
can - 0.0993797
MinMax:
cdr - 2.43089 *
can - 0.0978544
MaxAbs:
cdr - 1.95208 **
can - 0.0967865
no scale:
cdr - 1.61775 ***
can - 0.0993797
MinMax:
cdr - 2.27693 *
can - 0.105339
MaxAbs:
cdr - 1.96837 **
can - 0.0973526
Assign weights accroding to the clusters' sizes. Cluster weight = ln(cluster size) / ln(minimal cluster size)
. Clustering helps:
However prediction with clustering is a little bit off due to the small weights on big clusters:
??? ???
Factor 0.3, patience 3
Factor 0.1, patience 3
Factor 0.1, patience 6
??? (to add noise)
???