LFM2-ColBERT-350M

350M parameter ColBERT model for multi-language document retrieval and reranking

← Back to Liquid Nanos

LFM2-ColBERT-350M generates dense embeddings for document retrieval and reranking using the ColBERT late-interaction architecture. It supports 8 languages and excels at semantic search tasks.

HF

Specifications

Property	Value
Parameters	353M
Context Length	32K tokens
Document Length	512 tokens
Query Length	32 tokens
Output Dimensionality	128
Similarity Function	MaxSim

Supported Languages: English, Arabic, Chinese, French, German, Japanese, Korean, Spanish

Semantic Search

Multi-language retrieval

Reranking

Score retrieved documents

RAG Pipeline

Retrieval for generation

Quick Start

This model uses PyLate for inference, not standard Transformers or llama.cpp.

PyLate

Install:

pip install -U pylate

Indexing Documents:

from pylate import indexes, models, retrieve

# Load the ColBERT model
model = models.ColBERT(
    model_name_or_path="LiquidAI/LFM2-ColBERT-350M",
)
model.tokenizer.pad_token = model.tokenizer.eos_token

# Initialize the PLAID index
index = indexes.PLAID(
    index_folder="pylate-index",
    index_name="index",
    override=True,
)

# Encode and index documents
documents_ids = ["1", "2", "3"]
documents = ["document 1 text", "document 2 text", "document 3 text"]

documents_embeddings = model.encode(
    documents,
    batch_size=32,
    is_query=False,
    show_progress_bar=True,
)

index.add_documents(
    documents_ids=documents_ids,
    documents_embeddings=documents_embeddings,
)

Retrieving Documents:

# Initialize retriever
retriever = retrieve.ColBERT(index=index)

# Encode queries
queries_embeddings = model.encode(
    ["query for document 3", "query for document 1"],
    batch_size=32,
    is_query=True,
    show_progress_bar=True,
)

# Retrieve top-k documents
scores = retriever.retrieve(
    queries_embeddings=queries_embeddings,
    k=10,
)

Reranking:

from pylate import rank, models

queries = ["query A", "query B"]
documents = [
    ["document A", "document B"],
    ["document 1", "document C", "document B"],
]
documents_ids = [[1, 2], [1, 3, 2]]

model = models.ColBERT(
    model_name_or_path="LiquidAI/LFM2-ColBERT-350M",
)

queries_embeddings = model.encode(queries, is_query=True)
documents_embeddings = model.encode(documents, is_query=False)

reranked_documents = rank.rerank(
    documents_ids=documents_ids,
    queries_embeddings=queries_embeddings,
    documents_embeddings=documents_embeddings,
)