search

aqlaboratory / proteinnet

Standardized data set for machine learning of protein structure
MIT License
867 stars 132 forks source link

Duplicate training samples with different coordinates, PSSM and entropy #26

Open tueboesen opened 4 years ago

tueboesen commented 4 years ago

During a sanity check of this data I noticed that quite a lot of the training examples have identical sequences, but with different PSSM and entropy. The coordinates for these duplicates are also not identical, even under translation/rotation, though the one example I actually plotted after matching the coordinates under translation and rotation, had coordinates that we close to identical, but deviated in a few places.

See the attached example (it was too long to paste in here) identical_sequences.zip

Other training examples were repeated 6 times in the data.

Is there any good reason for this or is this an error?

tueboesen commented 4 years ago

It should be noted that this problem is not just in the training_100 data, but actually also extends into the training_95 data. I find this very surprising since I would have expected the clustering to at the very least group/remove identical sequences.