> Bio-engineering & bioinformatics pipelines > Protein Language Modeling > Engineer Protein Function Predictors with Embeddings in Python

Protein Language Modeling

Engineer Protein Function Predictors with Embeddings in Python

Engineer Protein Function Predictors with Embeddings in Python

The era of high-throughput sequencing has flooded us with an unprecedented volume of protein data, creating both a challenge and an immense opportunity. Understanding protein function is paramount for drug discovery, bio-engineering, and fundamental biological research, yet experimental determination remains slow and costly. A new paradigm has emerged: leveraging the latent information encoded within protein sequences through sophisticated machine learning. Specifically, protein embeddings, dense numerical representations derived from advanced neural networks, have revolutionized our capacity to infer function computationally.


These embeddings capture complex evolutionary and structural relationships, transforming raw sequences into rich, actionable features. This article empowers you to activate these powerful embeddings in Python, forging robust models that predict protein functionality with surgical precision. We decode the entire pipeline, from preparing your data to training and evaluating predictive models, ensuring you harness the full potential of this cutting-edge approach. Prepare to transform raw biological data into profound functional insights. This journey builds directly upon the foundational advancements that allow us to leverage cutting-edge AI for profound insights into protein sequences and their resulting embeddings, making the subsequent prediction tasks feasible and highly effective.

Published on:
Activate Protein Embeddings: The Foundation of Functional Inference

Activate Protein Embeddings: The Foundation of Functional Inference

Protein embeddings represent a monumental leap in how we analyze biological data. These are dense vector representations, typically generated by deep learning models like Transformers (e.g., ESM-2, ProtT5), that encode the biochemical, structural, and evolutionary context of a protein sequence into a numerical space. Each dimension within the vector captures a nuanced feature, far surpassing the simplistic one-hot encoding or physicochemical properties traditionally used. By transforming a variable-length protein sequence into a fixed-size vector, embeddings unlock the full power of classical and deep learning algorithms, allowing them to process and learn from complex biological information with unprecedented efficiency.


We embrace embeddings because they encapsulate a protein's 'language' – its evolutionary history, potential interactions, and functional motifs – in a format readily consumable by machine learning models. This eliminates the arduous task of manual feature engineering, which often misses subtle yet critical biological signals. The choice of embedding model profoundly impacts downstream predictions; contextual embeddings, which generate different representations for the same amino acid based on its surrounding sequence, generally outperform non-contextual ones by capturing dynamic biological relevance. To commence our journey, we must first activate these embeddings by loading them into our Python environment. We often find these pre-computed, stored in formats like HDF5 or CSV, ready to be mapped to their corresponding protein identifiers and functional labels.


The provided Python code illustrates how to load these crucial embedding files. We prioritize robust loading mechanisms, ensuring that whether your data is in a hierarchical data format (HDF5) or a tabular structure (CSV), you can seamlessly integrate it. A common practice involves storing embeddings as a dictionary where protein IDs serve as keys, facilitating rapid lookup and alignment with experimental labels. This initial step is surgical; incorrect loading or misalignment can cripple the entire predictive pipeline. We validate the integrity of our loaded embeddings by checking their dimensions and ensuring a one-to-one mapping with our protein identifiers, solidifying the foundation for all subsequent analyses and predictions.

import numpy as np
import pandas as pd
import h5py

def load_embeddings_from_h5(filepath):
    """Loads protein embeddings from an HDF5 file."""
    try:
        with h5py.File(filepath, 'r') as f:
            # Assuming embeddings are stored under a key, e.g., 'embeddings'
            # And protein IDs are under 'protein_ids'
            embeddings = {key: f[key][()] for key in f.keys() if 'id' not in key}
            protein_ids = list(f['protein_ids'][()]) if 'protein_ids' in f else None # Example
            return embeddings, protein_ids
    except Exception as e:
        print(f"Error loading HDF5 file: {e}")
        return None, None

def load_embeddings_from_csv(filepath, id_col='protein_id', embedding_cols_prefix='dim_'):
    """Loads protein embeddings from a CSV file."""
    try:
        df = pd.read_csv(filepath)
        protein_ids = df[id_col].tolist()
        # Identify embedding columns dynamically or by prefix
        embedding_cols = [col for col in df.columns if col.startswith(embedding_cols_prefix)]
        embeddings_array = df[embedding_cols].values
        
        # Convert to a dictionary for easier lookup by ID if desired
        embeddings_dict = {pid: emb for pid, emb in zip(protein_ids, embeddings_array)}
        return embeddings_dict, protein_ids
    except Exception as e:
        print(f"Error loading CSV file: {e}")
        return None, None

# --- Example Usage (Simulated Data) ---
# For demonstration, let's create dummy embeddings
# In a real scenario, these would come from ESM, ProtT5, etc.
# A dictionary mapping protein IDs to their embedding vectors

# Simulate generating/loading embeddings
def generate_dummy_embeddings(num_proteins=10, embedding_dim=128):
    dummy_embeddings = {}
    dummy_ids = [f"PROT_{i:04d}" for i in range(num_proteins)]
    for pid in dummy_ids:
        dummy_embeddings[pid] = np.random.rand(embedding_dim) # Random vector
    return dummy_embeddings, dummy_ids

print("Generating dummy embeddings...")
protein_embeddings, protein_ids_list = generate_dummy_embeddings(num_proteins=5, embedding_dim=1024)

print(f"Loaded {len(protein_embeddings)} protein embeddings.")
print(f"Example Protein ID: {list(protein_embeddings.keys())[0]}")
print(f"Embedding dimension: {protein_embeddings[list(protein_embeddings.keys())[0]].shape[0]}")

# Example of how to access an embedding:
# specific_protein_id = "PROT_0001"
# if specific_protein_id in protein_embeddings:
#     embedding_vector = protein_embeddings[specific_protein_id]
#     print(f"Embedding for {specific_protein_id}: {embedding_vector[:5]}...") # Show first 5 elements

Forge Your Dataset: Preparing Embeddings and Labels for Training

With our protein embeddings activated, the next critical phase involves forging a robust dataset suitable for machine learning. This requires the meticulous alignment of embedding vectors with their corresponding functional labels. Functional labels are often curated from public databases like GO (Gene Ontology) or experimentally derived annotations. We ensure that for every protein ID, we have both a valid embedding and an accurate functional classification. This synchronization is pivotal; any mismatch or missing data can introduce noise or bias, compromising the predictive power of our models.


Data preparation extends beyond mere alignment. Protein function prediction can manifest in various forms: binary classification (e.g., enzyme vs. non-enzyme), multi-class classification (e.g., classifying into specific enzyme classes), or even multi-label classification (where a single protein can have multiple functions). Our approach must adapt to the problem type. For multi-class scenarios, we use integer encoding for labels. For multi-label, a binary matrix representation is typically employed. This article primarily focuses on multi-class classification, a common and representative task.


Once aligned, we engineer our dataset into features (X) and target labels (y). The features are the numerical protein embeddings, typically stacked into a NumPy array, where each row represents a protein and each column an embedding dimension. The labels form another NumPy array. We then surgically split this dataset into training and testing sets. A standard 80/20 split is often applied, reserving 20% of the data for unbiased model evaluation. Crucially, we employ stratified sampling, particularly for imbalanced datasets, to ensure that the proportion of each functional class is maintained across both the training and testing partitions. This prevents the training set from being skewed towards dominant classes, which could lead to models that perform poorly on rarer but equally important functions. This meticulous data preparation phase dictates the ceiling of our model's performance; a well-prepared dataset is the bedrock of reliable predictions.

import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split

# --- Re-use dummy embeddings from previous step ---
# protein_embeddings = { ... } # A dictionary mapping protein_id to embedding vector
# protein_ids_list = [ ... ] # List of all protein IDs

# Simulate loading protein function labels
def generate_dummy_labels(protein_ids, num_functions=3):
    dummy_labels = {}
    # Assign a random primary function (0, 1, or 2) to each protein
    for pid in protein_ids:
        dummy_labels[pid] = np.random.randint(0, num_functions)
    return dummy_labels

print("Generating dummy function labels...")
protein_function_labels = generate_dummy_labels(protein_ids_list, num_functions=3)
print(f"Loaded {len(protein_function_labels)} protein function labels.")

# Combine embeddings with labels
def prepare_dataset(embeddings_dict, labels_dict):
    X = [] # Features (embeddings)
    y = [] # Target labels
    common_protein_ids = []

    # Ensure consistent ordering and only use proteins with both embedding and label
    for pid in sorted(embeddings_dict.keys()): # Sort for reproducibility
        if pid in labels_dict:
            X.append(embeddings_dict[pid])
            y.append(labels_dict[pid])
            common_protein_ids.append(pid)
    
    if not X or not y:
        raise ValueError("No common proteins found between embeddings and labels. Check your data.")

    return np.array(X), np.array(y), common_protein_ids

print("Preparing dataset from embeddings and labels...")
features, labels, common_pids = prepare_dataset(protein_embeddings, protein_function_labels)

print(f"Dataset prepared: {features.shape[0]} samples, {features.shape[1]} features (embedding dimensions).")
print(f"Labels shape: {labels.shape}")

# Split data into training and testing sets
# We maintain the 80/20 split as a common best practice
# 'stratify=labels' ensures that the proportion of target classes is roughly the same in both the training and test sets.
print("Splitting data into training and testing sets (80/20 split)...")
X_train, X_test, y_train, y_test = train_test_split(
    features, labels, test_size=0.2, random_state=42, stratify=labels
)

print(f"Training set size: {X_train.shape[0]} samples, {y_train.shape[0]} labels")
print(f"Testing set size: {X_test.shape[0]} samples, {y_test.shape[0]} labels")

# Verify label distribution in train and test sets (optional, good for checking stratification)
# print("Train label distribution:", pd.Series(y_train).value_counts(normalize=True))
# print("Test label distribution:", pd.Series(y_test).value_counts(normalize=True))
Engineer Predictive Models: Training on Protein Embeddings

Engineer Predictive Models: Training on Protein Embeddings

With our dataset meticulously prepared, we now engineer the core of our predictive system: the machine learning models. The versatility of protein embeddings allows us to deploy a wide array of algorithms, from classical statistical models to advanced neural networks. The choice of model is not arbitrary; it depends on the complexity of the functional prediction task, the size of our dataset, and the computational resources available. We investigate several robust classifiers commonly used in bioinformatics: Random Forest Classifiers, which excel at handling high-dimensional data and provide insights into feature importance; Logistic Regression, a foundational linear model for quick baselines; and Multi-layer Perceptrons (MLPs), simple neural networks capable of learning non-linear relationships, making them powerful for complex biological patterns.


Training these models transcends a mere 'fit' operation. We activate a process of systematic optimization, primarily through hyperparameter tuning. Hyperparameters, such as the number of estimators in a Random Forest or the learning rate in an MLP, are external configurations that significantly influence model performance. We employ techniques like Grid Search with Cross-Validation to exhaustively explore a predefined space of hyperparameter combinations. Cross-validation, typically k-fold (e.g., 3 or 5-fold), partitions the training data to validate models iteratively, ensuring our chosen hyperparameters lead to generalized performance, not just good performance on a single training split. This rigorous approach prevents overfitting, a critical pitfall where a model learns the training data too well, failing to generalize to unseen proteins.


The Python code provided showcases the implementation of these training strategies using `scikit-learn`, a cornerstone library for machine learning. We define a set of candidate models, each paired with a grid of hyperparameters. The `GridSearchCV` utility automates the cross-validation and hyperparameter search, identifying the optimal configuration that maximizes a chosen performance metric (e.g., accuracy, F1-score for imbalanced data). This systematic exploration is crucial; it transforms guesswork into a data-driven optimization process. We prioritize model simplicity where possible, only escalating to more complex architectures when the biological signal demands it. This phase is about forging a model that not only predicts accurately but also does so efficiently and reliably on new, unseen protein sequences.

import numpy as np
from sklearn.ensemble import RandomForestClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.neural_network import MLPClassifier
from sklearn.svm import SVC
from sklearn.model_selection import GridSearchCV
from sklearn.metrics import accuracy_score
import warnings

warnings.filterwarnings('ignore') # Suppress warnings for cleaner output

# --- Re-use prepared data from previous step ---
# X_train, X_test, y_train, y_test = ...

# Define a list of models to experiment with
# We select robust, commonly used classifiers that handle high-dimensional data well.
# For each model, we specify a dictionary of hyperparameters for GridSearchCV.

models_to_train = {
    "RandomForestClassifier": {
        "model": RandomForestClassifier(random_state=42),
        "params": {
            'n_estimators': [50, 100], # Number of trees in the forest
            'max_depth': [10, 20]     # Maximum depth of the tree
        }
    },
    "LogisticRegression": {
        "model": LogisticRegression(max_iter=1000, random_state=42, solver='liblinear'), # 'liblinear' for small datasets
        "params": {
            'C': [0.1, 1.0, 10.0]     # Inverse of regularization strength
        }
    },
    "MLPClassifier": {
        "model": MLPClassifier(random_state=42, max_iter=500), # Multi-layer Perceptron (Neural Network)
        "params": {
            'hidden_layer_sizes': [(64,), (128, 64)], # Tuple of integers, i-th element represents the number of neurons in the i-th hidden layer.
            'activation': ['relu']                     # Activation function for the hidden layer
        }
    }
    # Support Vector Machine (SVC) can be added but might be slow for very high-dimensional data
    # "SVC": {
    #     "model": SVC(random_state=42, probability=True),
    #     "params": {
    #         'C': [0.1, 1.0],
    #         'kernel': ['rbf']
    #     }
    # }
}

def train_and_evaluate_model(model_name, model_info, X_train, y_train, X_test, y_test):
    print(f"\n--- Training {model_name} ---")
    
    # Use GridSearchCV for hyperparameter tuning and cross-validation
    # cv=3 means 3-fold cross-validation
    # scoring='accuracy' can be replaced with 'f1_weighted' or 'roc_auc_ovr' for imbalanced data
    grid_search = GridSearchCV(model_info["model"], model_info["params"], cv=3, scoring='accuracy', n_jobs=-1, verbose=1)
    
    print(f"Performing Grid Search for {model_name}...")
    grid_search.fit(X_train, y_train)
    
    best_model = grid_search.best_estimator_
    print(f"Best parameters for {model_name}: {grid_search.best_params_}")
    print(f"Best cross-validation score: {grid_search.best_score_:.4f}")
    
    # Evaluate on the test set
    y_pred = best_model.predict(X_test)
    test_accuracy = accuracy_score(y_test, y_pred)
    print(f"Test Accuracy for {model_name}: {test_accuracy:.4f}")
    
    return best_model, test_accuracy

# Store results
best_models = {}
model_test_accuracies = {}

# Iterate and train each model
for name, info in models_to_train.items():
    model, accuracy = train_and_evaluate_model(name, info, X_train, y_train, X_test, y_test)
    best_models[name] = model
    model_test_accuracies[name] = accuracy

print("\n--- Training Summary ---")
for name, accuracy in model_test_accuracies.items():
    print(f"Model: {name}, Test Accuracy: {accuracy:.4f}")
    
# You can now use best_models['RandomForestClassifier'] for further analysis or deployment
# For example, to make predictions:
# new_protein_embedding = np.random.rand(1, X_train.shape[1]) # A single new embedding
# predicted_function = best_models['RandomForestClassifier'].predict(new_protein_embedding)
# print(f"Predicted function for new protein: {predicted_function[0]}")

Decode Performance: Evaluating and Interpreting Protein Function Predictions

Once our predictive models are engineered, the crucial step is to decode their performance and interpret their implications for protein function prediction. We move beyond simple accuracy, embracing a suite of metrics that paint a comprehensive picture of model efficacy, especially for biological datasets which are often imbalanced. Key metrics include Precision (the proportion of true positives among all positive predictions), Recall (the proportion of true positives among all actual positives), and the F1-score (the harmonic mean of precision and recall), which is particularly vital for assessing performance on minority classes. For multi-class problems, we use weighted averages of these metrics to account for class imbalance.


Beyond these, the Receiver Operating Characteristic (ROC) curve and its associated Area Under the Curve (AUC) provide a robust measure of a model's ability to distinguish between classes across various classification thresholds. For multi-class scenarios, we typically compute a One-vs-Rest (OvR) AUC. A perfect classifier achieves an AUC of 1.0. We also construct a Confusion Matrix, a visual tool that breaks down correct and incorrect predictions for each class. This matrix instantly reveals where a model excels and, more importantly, where it struggles, highlighting specific functions that are frequently misclassified or confused with others, offering actionable insights for model refinement.


The provided Python code leverages `scikit-learn`'s extensive metrics module to surgically calculate these performance indicators. We also integrate `matplotlib` and `seaborn` to visualize the confusion matrix, transforming raw numbers into intuitive graphical representations. Interpretation extends beyond just numbers; we ask critical biological questions: Are misclassifications biologically plausible? Does the model show bias towards certain protein families or functions? We validate our models not just on statistical robustness but on biological relevance. Common pitfalls include over-reliance on accuracy alone for imbalanced datasets, neglecting the interpretability of misclassifications, or failing to understand the biological context of false positives and false negatives. By rigorously evaluating and interpreting our model's performance, we ensure our protein function predictions are not only statistically sound but also biologically meaningful, propelling us closer to conquering biological frontiers with computational precision.

import numpy as np
import pandas as pd
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score, roc_auc_score, confusion_matrix, classification_report
import matplotlib.pyplot as plt
import seaborn as sns

# --- Re-use trained models and test data from previous step ---
# best_models = { ... } # Dictionary of best trained models
# X_test, y_test = ... # Test features and labels

def evaluate_classifier(model, X_test, y_test, model_name):
    """Evaluates a classifier model and prints key metrics."""
    print(f"\n--- Evaluation for {model_name} ---")
    y_pred = model.predict(X_test)
    
    # For multi-class classification, use 'weighted' or 'macro' for average metrics
    # 'weighted' accounts for class imbalance, 'macro' treats all classes equally.
    accuracy = accuracy_score(y_test, y_pred)
    precision = precision_score(y_test, y_pred, average='weighted', zero_division=0)
    recall = recall_score(y_test, y_pred, average='weighted', zero_division=0)
    f1 = f1_score(y_test, y_pred, average='weighted', zero_division=0)
    
    print(f"Accuracy: {accuracy:.4f}")
    print(f"Precision (weighted): {precision:.4f}")
    print(f"Recall (weighted): {recall:.4f}")
    print(f"F1-Score (weighted): {f1:.4f}")
    
    # ROC AUC for multi-class requires 'predict_proba' and 'ovr' (One-vs-Rest)
    # Check if the model has predict_proba method
    if hasattr(model, "predict_proba"):
        y_proba = model.predict_proba(X_test)
        # roc_auc_score requires one-hot encoded true labels for multi-class 'ovr'
        from sklearn.preprocessing import LabelBinarizer
        lb = LabelBinarizer()
        y_test_binarized = lb.fit_transform(y_test)
        
        # Handle cases with single class in y_test if LabelBinarizer fails
        if y_test_binarized.shape[1] == 1: # If only one class present after binarization
            print("Cannot compute ROC AUC for multi-class with only one class in test set.")
        else:
            roc_auc = roc_auc_score(y_test_binarized, y_proba, multi_class='ovr', average='weighted')
            print(f"ROC AUC (OVR, weighted): {roc_auc:.4f}")
    else:
        print("Model does not support `predict_proba` for ROC AUC calculation.")

    print("\nClassification Report:")
    print(classification_report(y_test, y_pred, zero_division=0))
    
    # Plot Confusion Matrix
    cm = confusion_matrix(y_test, y_pred)
    plt.figure(figsize=(8, 6))
    sns.heatmap(cm, annot=True, fmt='d', cmap='Blues', 
                xticklabels=model.classes_ if hasattr(model, 'classes_') else np.unique(y_test), 
                yticklabels=model.classes_ if hasattr(model, 'classes_') else np.unique(y_test))
    plt.title(f'Confusion Matrix for {model_name}')
    plt.xlabel('Predicted Label')
    plt.ylabel('True Label')
    plt.show()
    
    return y_pred

# Assuming 'best_models' from the previous step is available
# Let's evaluate the RandomForestClassifier as an example
if 'RandomForestClassifier' in best_models:
    print(f"\nExecuting evaluation for the best RandomForestClassifier...")
    rf_model = best_models['RandomForestClassifier']
    rf_predictions = evaluate_classifier(rf_model, X_test, y_test, "RandomForestClassifier")

if 'LogisticRegression' in best_models:
    print(f"\nExecuting evaluation for the best LogisticRegression...")
    lr_model = best_models['LogisticRegression']
    lr_predictions = evaluate_classifier(lr_model, X_test, y_test, "LogisticRegression")

if 'MLPClassifier' in best_models:
    print(f"\nExecuting evaluation for the best MLPClassifier...")
    mlp_model = best_models['MLPClassifier']
    mlp_predictions = evaluate_classifier(mlp_model, X_test, y_test, "MLPClassifier")

Key Takeaways

Protein Embeddings: The Biological Feature Revolution

Protein embeddings are dense numerical representations derived from advanced AI models, encapsulating deep biochemical, structural, and evolutionary information of protein sequences. They transform variable-length sequences into fixed-size vectors, making them ideal features for machine learning models. Activating these embeddings is the first critical step, often involving loading from HDF5 or CSV files, ensuring correct mapping to protein IDs.

Dataset Forging: Precision Alignment is Key

Preparing the dataset involves meticulously aligning protein embeddings with their corresponding functional labels (e.g., from GO terms). This process demands careful synchronization to avoid data mismatches. Data is then structured into features (X) and labels (y), followed by a stratified train-test split (e.g., 80/20) to ensure representative distribution of functional classes, especially vital for imbalanced datasets.

Model Engineering: From Baselines to Neural Networks

We deploy various machine learning models (e.g., Random Forest, Logistic Regression, MLP Classifiers) using Python's scikit-learn. The core of this phase is systematic optimization via hyperparameter tuning and cross-validation (e.g., using GridSearchCV). This rigorous approach identifies optimal model configurations, prevents overfitting, and ensures the model generalizes well to unseen data, forging robust predictive capabilities.

Performance Decoding: Comprehensive Evaluation for Biological Relevance

Model evaluation extends beyond accuracy. We decode performance using a suite of metrics: Precision, Recall, F1-score (weighted for imbalance), and ROC AUC (One-vs-Rest). A Confusion Matrix visually highlights misclassification patterns. This comprehensive approach, combined with biological interpretation of results, ensures that our predictions are not only statistically sound but also biologically meaningful, guiding further research and application.

Editorial Team
← Engineer Protein Transformers: Python Deep Learning Pipeline Map Protein Space: Cluster & Visualize Embeddings with Python →

FAQ

  • What are protein embeddings and why are they crucial for function prediction?

    Protein embeddings are dense numerical vectors that capture the complex biochemical, structural, and evolutionary information of a protein sequence. They are crucial because they transform variable-length sequences into fixed-size, rich feature representations, enabling machine learning models to effectively learn and predict protein functions without extensive manual feature engineering. These embeddings encapsulate the 'language' of proteins, making sophisticated functional inference possible.

  • How do I choose the right machine learning model for protein function prediction?

    Model selection depends on your dataset size, complexity of the function, and computational resources. For high-dimensional data from embeddings, Random Forest Classifiers are robust. Logistic Regression offers a good baseline. For more complex, non-linear patterns, Multi-layer Perceptrons (MLPs) or deeper neural networks are effective. Begin with simpler models and escalate complexity only if the data demands it, always prioritizing interpretability and efficiency.

  • What are common pitfalls in training models on protein embeddings?

    Common pitfalls include data misalignment (embeddings not matching labels), ignoring class imbalance in functional categories (leading to biased models), overfitting (where a model performs well on training data but poorly on new data), and over-reliance on single metrics like accuracy. Always use stratified sampling, hyperparameter tuning with cross-validation, and a comprehensive set of evaluation metrics (Precision, Recall, F1, ROC AUC, Confusion Matrix) to build robust models.

  • How can I interpret my protein function prediction model's results beyond basic accuracy?

    Beyond accuracy, activate a multi-faceted evaluation strategy. Use Precision, Recall, and F1-score to understand performance across individual classes, especially for imbalanced datasets. Analyze the Confusion Matrix to identify specific misclassification patterns. Compute the ROC AUC to assess overall discriminative power. Most critically, interpret results in a biological context: Are misclassifications biologically plausible? Does the model align with known biological mechanisms?

  • Where do I obtain protein embeddings for my prediction task?

    Protein embeddings are typically generated by pre-trained deep learning models. Popular choices include: ESM (Evolutionary Scale Modeling) embeddings (e.g., ESM-2), ProtT5 embeddings, or embeddings from other large protein language models. You can often find pre-computed embeddings for large public datasets or generate them yourself using available model weights and inference scripts for your specific proteins. We leverage these powerful, pre-trained representations to jumpstart our functional prediction pipelines.