Rui Jiang - Feb 09, 2025
AI chatbots are rapidly transforming e-commerce, but most fail to provide a great customer experience. Why? Because they struggle to understand products accurately, handle a wide variety of customer inquiries, and adapt to store-specific needs.
Most AI chatbots today follow a basic pattern:
This works for simple questions, but it often fails in real-world e-commerce scenarios. From poor product recommendations to inaccurate order tracking responses, many chatbots create frustration rather than solving problems.
In this post, we’ll explore four major challenges when building an AI chatbot for Shopify and how to overcome them.
One major challenge is that vector search alone does not always return the most relevant products. Most stores have between 10 and 100 products, but successful stores may carry thousands or even tens of thousands of items. This creates difficulties in ensuring accurate search results.
For example, in a snow sports store, searching for “snowboard bindings” using OpenAI’s embedding model (`text-embedding-3-small`) might return the following top 5 results (with the distance):
While some of these are relevant, others (like ski bindings) should not be prioritized over snowboard bindings. Vector search based retrieval struggles with categorical precision.
Similarly, vector search struggles with model numbers. In a watch parts store, a customer looking for “Ruber strap for 7S26-0020” may not find the correct product, even though it exists in the catalog. This is because vector search does not perform well on alphanumeric model numbers.
At Shopily, we combine vector search with keyword-based retrieval to improve accuracy. This approach ensures that customers find exactly what they are looking for by:
Unlike large platforms like Amazon, most Shopify stores have limited historical sales data and fewer products in each category. Standard ranking methods like Navboost (boosting engaged products in search results) are ineffective due to sparse data. Also as third-party app developers, we only have very limited access to historical data. These limitations make it challenging to apply traditional ranking algorithms that rely heavily on past user interactions.
With LLMs, we can significantly improve ranking strategies:
One major challenge in this approach is latency—we need to ensure that LLM-powered enhancements do not slow down the chatbot. By optimizing query processing and caching, we maintain fast response times while improving accuracy.
Initially, we categorized questions into predefined types and used specific prompts for each. However, this approach quickly became unmanageable because customer questions often fall into multiple categories. For example, a customer viewing a product page might ask for other product recommendations or order status, which does not fit neatly into “product information”
Instead of using rigid query classification, we developed a single, flexible prompt that can handle multiple types of customer inquiries. Our AI:
Each store has unique preferences regarding product recommendations, store policies, and customer interactions. A generic AI chatbot cannot accurately represent a business’s brand and expertise.
For example, a gaming PC store might want its AI to recommend products based on store's gaming PC configurations.
With Shopily AI, merchants can add store-specific knowledge that the chatbot follows when responding to customers. Additionally:
Building an effective AI chatbot for Shopify is not just about using the latest LLM—it requires thoughtful design to overcome real-world challenges. At Shopily, we have:
These enhancements make our AI chatbot truly useful for Shopify store owners, providing better product discovery, customer support, and sales conversions.
Want to add Shopily AI Chatbot to your store? Try it out on our demo store and install it on your Shopify store today!