Embeddings

Embeddings are vector representations of data, particularly text, that capture semantic meaning and context with their placement in a high-dimensional vector space. These vectors are designed to preserve the content and meaning of the data, allowing similar pieces of data to have embeddings that are closer together compared to unrelated data. This facilitates various Natural Language Processing (NLP) tasks such as search, clustering, recommendations, anomaly detection, and classification. At Mixedbread, we offer state-of-the-art text embedding models. These embeddings are useful for a wide range of applications, enhancing the performance and accuracy of NLP solutions. For detailed information on how embeddings work and their applications, refer to our Embeddings Guide.

Reranking

Reranking is the process of sorting and prioritizing retrieved documents or information to ensure the most relevant data is used. It improves the quality and accuracy of information by evaluating and organizing retrieved items based on relevance. Reranking is crucial in Retrieval-Augmented Generation (RAG) frameworks, enhancing tasks like question answering and content generation by prioritizing the most pertinent information. See our Reranking Guide for details.

RAG

RAG is an AI framework that combines the capabilities of large language models (LLMs) with information retrieval systems. RAG operates in two main steps:

1. Retrieval: Relevant information is retrieved from a knowledge base using text embeddings stored in a vector store.
2. Generation: The retrieved information is inserted into the prompt for the LLM to generate a response.

RAG is useful for answering questions and generating content by leveraging external knowledge, including up-to-date and domain-specific information. This framework allows models to access and utilize information beyond their training data, reducing hallucination and improving factual accuracy.

Mixedbread offers models that are compatible with the RAG framework:

• Embeddings: Used to represent text in a high-dimensional vector space, enabling the retrieval of relevant information from a knowledge base.
• Reranking: Enhances the retrieval step by sorting and prioritizing the retrieved documents or pieces of information, ensuring the most relevant data is used for generation.
• ColBERT: An advanced retrieval model that enhances the accuracy and efficiency of the retrieval step, particularly useful for handling large-scale data and complex queries.

Tokens

Tokens are the smallest units processed by an embeddings or reranking model, representing sequences of characters like words or subwords. Text must be converted into tokens to be understood by a model. Each token is assigned a unique numerical index through a process called tokenization. For embeddings and reranking models, the input text must be shorter than the model's maximum context length. Model-specific limits are available in the model index.

Something Missing?

If you have any questions or need further clarification on any terms or concepts, please feel free to reach out to us. We're here to help you understand and leverage our models effectively.