mixedbread ColBERT
In this documentation, you'll learn all you need to know about the ColBERT architecture, a way to enable great reranking and retrieval performance without the computational needs of traditional cross-encoders.
What are the Traditional Approaches?
The typical search approach uses the same model to encode both documents and queries. We then choose a metric, such as cosine similarity, to measure the distance between the query and the documents. However, there is an issue with that: our model has to determine the optimal placement within the latent space, so that query and relevant documents are positioned closely together, but there is no interaction between query and document within the model.
On the other hand, we have models like cross-encoders. With cross-encoders, the query and documents are fed to the model together, improving search accuracy. Unfortunately, cross-encoders are extremely compute-intensive, since we need to pass all possible combinations of documents and queries to the model. Therefore, these models are not suitable for large-scale search and are mostly used for reranking.
What is the ColBERT Architecture?
ColBERT stands for Contextualized Late Interaction BERT, and it combines both vector search and cross-encoders. In ColBERT, the queries and the documents are first encoded separately. However, instead of creating a single embedding for the entire document, ColBERT generates contextualized embeddings for each token in the document. To search, the token-level query embeddings are compared with the token-level embeddings of the documents using the lightweight scoring function MaxSim. This allows ColBERT to capture more nuanced matching signals while still being computationally efficient. The resulting scores are then used to rank the documents based on their relevance to the query.
Similarity scoring process of query and document in a ColBERT model
While ColBERT can be used for both reranking and retrieval tasks, we mainly recommend using it for reranking and taking advantage of our powerful embedding model for retrieval-related use cases.
What's Next?
Now that you're familiar with the concept of ColBERT, get started with mxbai-colbert-large-v1!