Antibodies are particular Y-shaped proteins. They are one of the most essential parts of the adaptive immune system since they interact with antigens. Being able to engineer antibodies can significantly impact the development of therapeutics, where paratope prediction plays an essential role. Given the antibody's sequence, predicting the paratope means finding the specific amino acids that bind to its target. This repository provides the necessary tools to generate insights into paratope prediction, ultimately guiding model development towards a faster and better production of therapeutics.

	Alpha Fold Multimer Contacts	ESM2 650M + RF Transfer Learning
	6A0Z	6A0Z

Table of Contents

• Installation
• Dashboard
• Motivation
• Models
• Literature Datasets Summary
• Usage
• Vendored

Installation

The installation is currently supported with the usage of a conda environment.
Mambaforge installation is strongly encouraged:
github.com/conda-forge/miniforge

After having installed mamba or conda, clone the repository and create the environment:

# Clone it
git clone https://github.com/bayer-science-for-a-better-life/topefind-public

# Enter
cd topefind-public

# Create environment
mamba env create -f environment.yml

# Activate environment
mamba activate topefind

Dashboard

This repo comes with a dashboard made in Panel which is provided to analyze the results on a test set with some models. It can run on the fly on WASM, be aware that it might take some minutes to install all packages from micropip and fully open. The dashboard can be easily modified and new results can be visualized by cloning the repo and providing a new DataFrame.

Motivation

What can you do with better predictions?

• A more accurate prediction of the paratope, allows, e.g. the in-silico optimization of potential antibody candidates by first predicting the paratope and then optimizing on those specific amino acids of the paratope, or vice versa, keeping them fixed to enhance other properties such as immunogenicity, stability, and humanisation.

Why is it so difficult?

• The structure of complementarity-determining regions (CDRs) is not preserved, especially in HCDR3, resulting in an almost uniform random pattern to learn in those regions; and light chains have a reduced amount of paratope residues, ca. 5, as opposed to ca. 10 in the heavy chain, resulting in a greater imbalance setting.

• Predictions must be close-to-perfect in an out-of-distribution setting to assess the method's practicality. Consider mutating residues in non-paratope-predicted regions to optimize a property of the antibody while keeping the predicted paratope fixed: if the predictions were not to be close-to-perfect, the mutations would result in modifying the true paratope, disrupting affinity. Given the above-mentioned paratope size, one wrong mutation can be detrimental to affinity by 20% in light chains and 10\% in heavy chains. This is why the aim of the scoring metrics is much higher for a practical usage of the predictors in this setting/task.

Strange behaviour of current models and transfer learning

• The models reach a ceiling in performance on test sets (Expanded Paragraph Test Set).
• Surprisingly, the model capacity does not benefit significantly the downstream task, which contradicts downstream tasks behaviours on single protein properties, e.g. secondary structure prediction or contact prediction in ESM2 models.

Why is this happening?

• The models learn mostly the bivariate distribution of position and amino acid type (with some extra context of nearby residues).
• Training a random forest on top of such simple features highlights the problem.

• The bivariate distribution of absolute error given position and amino acid type assures that the models fail similarly:

Open questions?

• Is this a problem of the classifier only and not the feature extractor?
• If the classifier is the problem, can we overcome the problem by weighting the loss given the frequency of the amino acids to not let the model fail into a local minima?

Usage

The topefind package contains several subpackages:

• exploration: exploration code to answer questions (quick and dirty)
• dashboard: visualization of results
• vendored: vendored repositories

And several core modules:

• embedders.py: embedders for protein sequences
• predictors.py: paratope predictors (uses embedders)
• benchmarkers.py: benchmarks predictors (uses embedders and predictors)
• utils.py: utilities, plotting and necessary globals

Additionally:

• datasets is used to store datasets.
• models is used to store models.
• examples showcases usages.

Get a parsed and labelled version of SAbDab

To do this we need to use the data_hub.py.
This will create a local copy of SAbDab in datasets, and it will parse it into a convenient parquet file. It might take a while, currently SAbDab is ~6GB. Increase the number of jobs (n_jobs) for quicker parsing.

from topefind.data_hub import SabdabHub

df = SabdabHub(n_jobs=1)()

Create a paratope dataset to play with

# Select some columns of interest
df = df[[
    "pdb",
    "antibody_sequence",
    "antibody_imgt",
    "antibody_chain",
    "chain_type",
    "resolution",
    "scfv",
    "antigen_sequence",
    "antigen_chain",
    "antigen_type",
    "num_antigen_chains",
    "full_paratope_labels",
]]

# Let's filter according to some literature guidelines.
df = df.drop_duplicates("antibody_sequence")  # Don't bias the model.
df = df[(df.antibody_sequence.str.len() > 70) & (df.antibody_sequence.str.len() < 200)]  # Don't go < 70 ...ANARCI.
df = df[df.full_paratope_labels.apply(sum) >= 1]  # At least some positives.
df = df[(df.num_antigen_chains > 0) & (df.num_antigen_chains <= 3)]  # Follows the choice in Paragraph.
df = df[~df.scfv]  # Hard to deal with since two chains are connected.
df = df[df.antigen_type.isin(["protein", "peptide"])]
df = df[df.resolution < 3]  # Allows to define contacts above this resolution (used everywhere in literature).
df = df.reset_index()

# Done, a working dataset.
print(f"Dataset now contains {len(df)} entries")
print(f"{len(df[df.num_antigen_chains > 1])} entries are connected to multiple antigens")

sequences = df["antibody_sequence"].to_list()
labels = df["full_paratope_labels"].to_list()

test_sequence = sequences.pop()
test_label = labels.pop()

Let's train a classifier on top of ESM embeddings and predict the paratope

# Feel free to switch to ESM2 bigger models available under EmbedderName.
# Check `topefind.predictors` and `topefind.embedders`and choose your desired configuration.
import tabulate
import numpy as np
from joblib import Memory
from sklearn.ensemble import RandomForestClassifier

from topefind.predictors import PLMSKClassifier
from topefind.embedders import EmbedderName, ESMEmbedder

# Cache the model for further usage
memory = Memory("cache", verbose=0)
seed = 42

@memory.cache
def build_esm_rf(
        train_sequences,
        train_labels,
        emb_name=EmbedderName.esm2_8m,
        n_estimators=128
):
    return PLMSKClassifier(
        ESMEmbedder(emb_name),
        RandomForestClassifier(n_estimators=n_estimators, n_jobs=4, random_state=seed),
    ).train(train_sequences, train_labels)

esm_rf = build_esm_rf(sequences, labels)
preds = esm_rf.predict(test_sequence)
trues = np.array(test_label)
results = np.hstack(trues, preds).T

print(tabulate(results, headers=["labels", "predictions"], tablefmt="fancy_grid", floatfmt=".2f"))

Predict the paratope with an end-to-end model

from topefind.predictors import Parapred

parapred = Parapred()

preds = parapred.predict(test_sequence)
trues = np.array(test_label)
results = np.hstack(trues, preds).T

print(tabulate(results, headers=["labels", "predictions"], tablefmt="fancy_grid", floatfmt=".2f"))

Models

Some relevant models from the literature are reported for comparison.

^a check Vendored
^b topefind uses parapred-pytorch provided by alchemab. The original version in Keras is provided by eliberis and is not considered in topefind for inference.

Some relevant models in the literature are not included for the following reasons:
^c Weights not available.
^d Weights not available.
^e License not permissive enough.

Literature Datasets Summary

We report some relevant datasets and a brief summary of the pre-processing done to each one to get an overview of the current landscape. Check each model's paper reported in Models for more details.

^a Reported by Paragraph.
^b Reported by antiBERTa.
^c We use the parapred-pytorch adapted weights.
^d The script to download the dataset for pre-training the antibody LM is given but is not functional. The transfer-learning dataset is given but lacks the differentiation between heavy and light chains.
^e Accounts only in this case for the total number of antibody chains and not complexes.
^f Not explicitly stated by Parapred.

Vendored

Some related works are vendored for simplicity and reproducibility.
The vendored folder contains such software with its necessary LICENSE provided by the authors.
Weights of trained models are stored in each vendored package, if provided by the original authors.
We applied minor modifications to parts of the code to adapt for each use case.
Please refer to each package for more details.

Additionally, a small part of panel_chemistry is vendored for the dashboard.

Acknowledgements

Built with love ❤️ by Serban Cristian Tudosie during an internship at Bayer 🧬.
Several more people have contributed: Santiago Villalba, Pedro Reis, Adrien Bitton, Sven Giese.