How We Built a Smarter AI Chatbot for Shopify Stores (And What We Learned)

Rui Jiang - Feb 09, 2025

The Challenge of AI Chatbots in E-Commerce

AI chatbots are rapidly transforming e-commerce, but most fail to provide a great customer experience. Why? Because they struggle to understand products accurately, handle a wide variety of customer inquiries, and adapt to store-specific needs.

Most AI chatbots today follow a basic pattern:

1. Use vector search to fetch information from products and store pages.
2. Pass the retrieved content to an LLM to generate a response.

This works for simple questions, but it often fails in real-world e-commerce scenarios. From poor product recommendations to inaccurate order tracking responses, many chatbots create frustration rather than solving problems.

In this post, we’ll explore four major challenges when building an AI chatbot for Shopify and how to overcome them.

Challenge 1: Handling Large Product Catalogs

The Problem: Vector Search Limitations

One major challenge is that vector search alone does not always return the most relevant products. Most stores have between 10 and 100 products, but successful stores may carry thousands or even tens of thousands of items. This creates difficulties in ensuring accurate search results.

For example, in a snow sports store, searching for “snowboard bindings” using OpenAI’s embedding model (`text-embedding-3-small`) might return the following top 5 results (with the distance):

1. "Supermatic Snowboard Bindings - 2023/2024" (0.972)
2. "SX 7.5 GW Ski Bindings" (0.981)
3. "Race Skate Cross-Country Ski Bindings" (1.006)
4. "Aurora Snowboard Bindings - Women's - 2023/2024" (1.009)
5. "Vetta Snowboard Bindings - Women's - 2023/2024" (1.030)

While some of these are relevant, others (like ski bindings) should not be prioritized over snowboard bindings. Vector search based retrieval struggles with categorical precision.

Similarly, vector search struggles with model numbers. In a watch parts store, a customer looking for “Ruber strap for 7S26-0020” may not find the correct product, even though it exists in the catalog. This is because vector search does not perform well on alphanumeric model numbers.

The Solution: Hybrid Search (Vector + Keyword Matching)

At Shopily, we combine vector search with keyword-based retrieval to improve accuracy. This approach ensures that customers find exactly what they are looking for by:

• Using vector search to retrieve relevant products based on meaning: Rubber strap vs Silicon strap in product name.
• Using keyword search to capture exact matches: 7S26-0020
• Merging and re-ranking results to prioritize the most relevant options.

Challenge 2: Enhancing Product Ranking for Shopify Stores

The Problem: Lack of user engagement data

Unlike large platforms like Amazon, most Shopify stores have limited historical sales data and fewer products in each category. Standard ranking methods like Navboost (boosting engaged products in search results) are ineffective due to sparse data. Also as third-party app developers, we only have very limited access to historical data. These limitations make it challenging to apply traditional ranking algorithms that rely heavily on past user interactions.

The Solution: Leveraging LLMs for Relevance and Query Refinement

With LLMs, we can significantly improve ranking strategies:

• Query rewriting ensures that search terms better match product listings.
• Relevance evaluation helps refine search results dynamically.
• Custom ranking factors prioritize products based on store owner preferences.

One major challenge in this approach is latency—we need to ensure that LLM-powered enhancements do not slow down the chatbot. By optimizing query processing and caching, we maintain fast response times while improving accuracy.

Challenge 3: Handling a Wide Variety of Customer Questions

The Problem: Customer Queries Are Unpredictable

Customers ask a diverse range of questions, including:

• Product recommendations - “What’s the best snowboard for beginners?”
• Order status - “Where’s my package?”
• Returns & Refunds – “Can I return a product after 30 days?”
• Discount Codes – “Are there any promo codes available?”
• Product details - “Is this DEET free?”

Initially, we categorized questions into predefined types and used specific prompts for each. However, this approach quickly became unmanageable because customer questions often fall into multiple categories. For example, a customer viewing a product page might ask for other product recommendations or order status, which does not fit neatly into “product information”

The Solution: A Unified Prompt + AI-Powered Tool Integration

Instead of using rigid query classification, we developed a single, flexible prompt that can handle multiple types of customer inquiries. Our AI:

• Uses a unified LLM prompt to dynamically interpret any question.
• Integrates with store data to retrieve real-time order status, promotions, and policies.
• Acts as a command center, not just an information provider, utilizing the necessary tools to fetch the right response.

Challenge 4: Adapting to Store-Specific Knowledge

The Problem: Store-Specific Preferences Matter

Each store has unique preferences regarding product recommendations, store policies, and customer interactions. A generic AI chatbot cannot accurately represent a business’s brand and expertise.

For example, a gaming PC store might want its AI to recommend products based on store's gaming PC configurations.

The Solution: AI-Driven Customization

With Shopily AI, merchants can add store-specific knowledge that the chatbot follows when responding to customers. Additionally:

• Store owners only need to input knowledge to describe certain attributes customers should look for.
• The AI prioritize recommending products matching those attributes and aligning with business strategies.

Conclusion: Why Shopily AI Is a Smarter E-Commerce Chatbot

Building an effective AI chatbot for Shopify is not just about using the latest LLM—it requires  thoughtful design to overcome real-world challenges. At Shopily, we have:

• Implemented hybrid retrieval/ranking (vector + keyword search) for precise product discovery.
• Developed a flexible, unified AI prompt that handles diverse customer queries.
• Enabled store-specific knowledge customization to ensure responses align with business needs.
• Optimized performance to keep chatbot responses fast and relevant.

These enhancements make our AI chatbot truly useful for Shopify store owners, providing better product discovery, customer support, and sales conversions.

Want to add Shopily AI Chatbot to your store? Try it out on our demo store and  install it on your Shopify store today!