Providing truly relevant results is paramount in commerce search now that shoppers are increasingly using more conversational and complex queries. In these situations, where keyword-based search falls in recognizing the context of queries and the intent of shoppers, vector-based search comes into play.

Vector search identifies the semantic similarity between queries and products in your catalogue, returning more relevant product results faster.

Vector search is useful in certain cases where zero-result scenarios are more likely, helping to minimize those by offering results for semantically similar queries. Using vector similarity enables a powerful recommendation system that enriches results by creating a cascade of relevant products.

For example, if you search for the term “accent chair”, vector search displays results for similar searches like decorative search, patio chair, or rocking chair. The related results give you some relevant search options that are semantically similar to your initial search. Vector search achieves this by interpreting search queries and product-related information as vectors, the semantic neighbors of words.

Vector search vs keyword search

Vector search leverages artificial intelligence (AI), machine learning (ML), and natural language processing (NLP) techniques to capture the meaning and context of unstructured data. For a commerce search context, unstructured data applies to queries and product catalogue information.

Commerce search data is transformed into numerical representations called vectors. They're used to calculate the similarity or distance between data using approximate nearest neighbor (ANN) algorithms. So, similar products are placed close together and close to their corresponding queries in a high-dimensional vector space. All this process returns more precise and relevant product results faster to complex and conversational queries.

interact

To learn more about the ins and outs of the Empathy Platform Vector Search microservice to return vectorized results, check out the Microservices layer from the Interactive map.

Traditional keyword search instead is based on exact keyword matching, trying to match exact terms in queries with product catalogue information. If the precise search terms aren't included in the product catalogue, no results are returned. So, in many situations, traditional commerce search experiences need to be curated and complemented with features such as related tags, facets and filters, synonyms, or partial results.

In keyword-based search experiences, when shoppers are looking for something but they're not sure what it’s called, if they don’t have the exact search terms used in your product catalogue, they probably come to a dead end and leave searching. However, in a vector-based search experience, if shoppers don’t know the exact search terms but know what the product does or have a description, they probably find relevant results.

Vector search advantages

Vector search overcomes keyword matching constraints providing quick and accurate results to queries and meeting the shoppers at a more human level:

• Disambiguating term search context. It searches by and understands what shoppers mean, improving product relevance and discovery.
• Handling natural language queries. It’s good for fuzzy, broad, and conversational queries.
• Understanding synonyms and search term variations, like polysemy. It can retrieve products that match the intended meaning, even if different search terms or phrases are used.
• Improving UX. The inner search process is faster and more efficient.

Vector search limitations

Despite the benefits of vector search in commerce search experiences, keyword matching still rules for precise and simpler query search contexts.

For example, if a shopper searches for the query “cupcake” in a vector-based commerce search, the vectorized search engine will look for semantically similar queries such as muffins or even cinnamon rolls, croissants, and cookies, bringing results that probably are less relevant than keyword-matching results. Instead, in a traditional commerce search context, the query “cupcake” can return some accurate results like Vanilla and chocolate cupcakes, Mini iced white cupcakes, or Bakery fresh cupcakes, based on the matching of the search terms with the product catalogue.

note

Vector search goes beyond finding products, serving as an important tool for merchants to understand shopper intent and deliver a curated shopping experience.

Vector search in Empathy Platform

Empathy Platform’s vector search is built on a completely cloud-native stack, leveraging the capabilities of Kubernetes and Apache Spark to train semantic models effectively and create a computationally sustainable solution.

To ensure customer data and privacy are protected, Empathy Platform uses commerce-specific tagging events, which collect customer domain data under shoppers’ consent, and guarantee a personalized search experience that respects shoppers’ privacy. This customer domain-specific data enriches the open-source foundation semantic models to help the Empathy Platform Vector Search microservice understand the types and nature of the different datasets, without cross-domain jumping or using information from other customers, ensuring thus the integrity of customers' data.

interact

Curious about how data privacy is protected when training semantic models? Check the Semantic models in Empathy Platform.

Vector search + keyword search = hybrid search

In situations where traditional keyword-based search doesn’t account for the rich semantic information contained within the semantic data models, Empathy Platform leverages the benefits of AI-conversational search experiences to combine vector and keyword matching techniques in a hybrid solution. So, vector search in Empathy Platform extends and enriches the capabilities of keyword search to resolve no-results scenarios and offer responses to long-tail queries with vectorized product recommendations, or provide synonym suggestions, for example.

interact

Want to review the scenarios where vector search enhances keyword search? Check out Experience vector search.

Empathy Platform vector search complements keyword search by implementing consent-integral based vector models to add semantic information to the base keyword mechanisms and ensure that the hybrid-based search and discovery experiences are safe and private.

Empathy Platform’s vector search capabilities are becoming part of shoppers’ search experiences. They are seamlessly integrated into the shopping routines and eventually make up a unified index with keyword-based search as part of a unified, powerful search solution.

interact

Combining vector and keyword search mechanisms for a hybrid search experience is not that easy. Understand Empathy’s technical concerns and solutions in its way to hybrid search on Vector and keyword search as a unified index.

Privacy-minded hybrid search

Empathy Platform vector search complements keyword search by implementing consent-integral based vector models to add semantic information to the base keyword mechanisms and ensure that the hybrid-based search and discovery experiences are safe and integral.
Watch now this vector search recap and discover the path toward a privacy-integral hybrid search experience.