Inside AiChat: How Linguists Enhance Chatbot Performance and Accuracy

Ever wonder who is working behind a smart chatbot? Let’s speak with one of AiChat’s senior linguist, Amalia, to find out.

Q: May we have a quick introduction please, Amalia?

Hi, I’m Amalia, one of AiChat’s AI Linguists, responsible for training and maintaining our clients’ chatbots according to best practices. 

Q: For a start, what is the key metric that measures the effectiveness of AI chatbot?

As AI-powered chatbots continue to evolve, it is crucial to monitor and optimize their performance. One key metric for measuring their effectiveness is the chatbot containment rate. Containment rate is defined as the percentage of users who engage with a chatbot and then leave the conversation without speaking to a human agent. A “good” chatbot containment rate can vary based on the industry, the complexity of the tasks handled by the chatbot, and customer expectations. 

Nonetheless, a commonly referenced industry benchmark for chatbot containment rates is usually between 70% and 90%. A high containment rate indicates that the chatbot comprehends user queries, provides relevant information, and minimizes the need for human intervention. This not only enhances the user experience but also boosts overall operational efficiency. This is particularly important for businesses that utilize chatbots for customer service, support, or other interactive tasks. By tracking this metric, we can evaluate how effectively a chatbot operates within its designated scope and delivers accurate responses.

Q: In your opinion, what are the challenges in achieving a high containment rate?

Attaining a high containment rate brings its own set of challenges related to chatbot technology and user expectations. One significant challenge in achieving high containment rates is negative feedback from users. Negative feedback can adversely affect chatbot performance metrics and customer satisfaction scores. When users experience issues, they may leave negative feedback, impacting the bounce rate and overall satisfaction. To address this challenge, organizations should proactively collect and analyze feedback to pinpoint areas for improvement and enhance their chatbots.

Besides, while chatbots are proficient at handling routine topics, they may encounter difficulties with complex customer interactions or industry-specific problems. To address this, businesses should invest in enhancing the chatbot’s knowledge base and problem-solving abilities to better manage a broader spectrum of concerns.

Q: So, what is the role of the AI linguist?

In the context of human-AI collaboration, the involvement of linguists at AiChat is pivotal. Here’s how their contribution underscores the significance of this partnership:

1. Maintaining Knowledge Quality:

AiChat’s linguists ensure that the chatbot’s knowledge base is up-to-date and accurate. This maintenance is crucial for the chatbot to provide reliable and relevant information, enhancing user trust and satisfaction.

2. Understanding User Intentions:

By analyzing user inputs, linguists help the AI system understand and interpret user intentions accurately. This capability is essential for the chatbot to respond appropriately, making interactions more intuitive and effective.

3. Contextual and Meaningful Replies:

Linguists work on generating replies that are not only contextually relevant but also meaningful. This aspect of conversational design enhances the chatbot’s ability to engage users in natural and coherent dialogues, improving the overall user experience.

4. Improving Conversational Experience:

The expertise of linguists in structuring dialogues ensures that the chatbot can handle a wide range of queries with precision and empathy. This leads to a more engaging and satisfying interaction for users, bridging the gap between human-like conversation and automated responses.

By integrating the skills of linguists into chatbot development, AiChat exemplifies the strength of human-AI collaboration, showcasing how combining human expertise with AI capabilities can revolutionize conversational interfaces. This synergy not only enhances the functionality and user experience of chatbots but also sets a benchmark for future advancements in AI-driven communication technologies.

Q: What are AiChat’s Strategies in Improving the Chatbot Containment Rate?

1. Chatbot Training and Knowledge Base : We believe that weekly chatbot training is essential to enhance the user experience. It is crucial to ensure that the knowledge base is comprehensive and up-to-date so users receive relevant information. These are among the training processes carried out by linguists:

A. Diverse User Expressions:

• Train the chatbot with a minimum of 10 unique expressions for each type of query. These expressions should be sufficiently different to cover a wide range of user inputs.
• Example approach: Use variations in phrasing, synonyms, and different sentence structures.

B. Local Slangs:

• Include local slang and colloquial terms to ensure the chatbot can understand and respond to regional variations in language.

C. Post Go-Live Moderation:

• After the chatbot goes live, monitor its performance using an AI training dashboard.
• Regularly review user interactions to identify areas for improvement.

D. Out-of-Scope FAQs Compilation:

• Continuously compile out-of-scope questions to understand gaps in the knowledge base.
• Use these insights to update and expand the knowledge base.

E. User Expression Tracker:

• Utilize an expression tracker to monitor the effectiveness of trained expressions.
• Identify which expressions are working well and which need further refinement.

To optimize the entire process, as linguists, we would regularly review and update the training process based on feedback and performance data, ensure the language used by the chatbot is accurate and natural, and continuously expand the knowledge base to cover emerging topics and user queries.

2. Chatbot Design and User Interface : The success of a chatbot is highly dependent on its user interface, which functions as the gateway for user-bot interactions. To ensure an optimal experience, we follow a set of best practices when setting up a chatbot. We make sure our chatbots are interactive and engaging by incorporating call-to-action (CTA) buttons, setting an appropriate tonality, and testing for readability and overall user experience.

3. Leveraging Feedback : It is essential to consider additional metrics for a comprehensive evaluation of a chatbot. At AiChat, users have the opportunity to provide feedback directly within the chat interface, facilitating real-time communication and fostering a sense of engagement. Businesses can utilize direct feedback channels, such as a simple “Was this helpful?” prompt or a detailed feedback form, to gather valuable insights into user sentiment and preferences.

4. Regular Review of Performance Metrics : We believe it is important not to rely solely on the containment rate as a measure of success. To better assess chatbot effectiveness and user satisfaction, we integrate multiple metrics. While the containment rate is significant, it is essential to balance it with other performance indicators. At AiChat, we have an internal bot accuracy tracking system, which is calculated from the content management system’s training part, where we can validate, skip, and manage the data. The accuracy is measured by the ratio of correct responses to the total number of data points. Apart from that, we also analyze other various aspects such as CSAT scores, chatbot interactions time, and issues escalated to live chat support. These metrics provide valuable insights into the overall user experience.

Q: Can you tell us more about AiChat’s NLP?

NLP (Natural Language Processing) is instrumental in improving the containment rate. At AiChat, we utilize proprietary NLP technology to train a variety of user expressions and inquiries. For example, different expressions such as “Do you have delivery services?”, “Do you offer any part-time courses?”, and “I want to book a school tour” represent different intents. NLP helps recognize these intents, allowing bots to return accurate answers based on the chatbot’s knowledge base. Additionally, we have an Named-entity recognition (NER) feature that tags specific words or intents, enabling the AI to capture the user’s intent and provide contextual responses. This technology allows chatbots to correctly understand and interpret user inputs, resulting in more precise responses and enhanced service.

Q: Can you share with us a customer success story?

Sure, we have MR.DIY, one of the largest home improvement retailers worldwide, expanding its stores network in countries such as Malaysia, Singapore, Thailand, Brunei, Indonesia, Philippines, Cambodia, India, Turkey and Spain. MR.DIY has partnered with AiChat to roll out their Chatbots across the region, starting with MR.DIY Malaysia, followed by MR.DIY Indonesia, Thailand, and Philippines. We have helped MR.DIY chatbots to achieve containment rate of above 75% consistently for English and foreign languages such as Malay, Indonesian, Thai, and Tagalog. This is where NLP capabilities and chatbot training play an important role. It requires comprehensive language support and effective translation capabilities. A variety of user expressions for the same intent are continuously trained to ensure accurate understanding and responses.

Q: Can generative AI help in the containment rate?

The rise of AI has unlocked many revolutionary possibilities, including generative AI. At AiChat, our linguists are leveraging ChatGPT to generate user expressions, accelerating the training of our chatbots. Generative AI can create numerous variations of the same query, capturing the diverse ways users might phrase a question or request. This includes variations in tone, formality, slang, and regional dialects. For instance, instead of just training a bot to respond to “What’s the weather like?”, generative AI can create variations like “Is it going to rain today?”, “What’s the forecast?”, or “Do I need an umbrella?”, or in Singlish, “Got rain or not?”, “Weather how ah?”, or “Must bring brolly anot?”.

Q: To end, what can our readers take away from this?

In summary, the chatbot containment rate is a vital metric for assessing chatbot effectiveness and significantly impacts customer satisfaction. As chatbots gain prominence across various industries, understanding and measuring containment rates becomes increasingly important. AI linguists play a crucial role in optimizing chatbot performance through thorough training, advanced NLP techniques, intuitive design, and the use of user feedback. Additionally, the generative AI approach will continue to enhance chatbots’ language understanding. These advancements will enable chatbots to better understand user queries, resulting in higher containment rates and more accurate responses.

Thanks Amalia for your insightful sharing today. We look to hear more from you soon.

Are you looking to enhance your chatbot solutions? We take pride in our chatbots’ high containment rates and are constantly seeking ways to improve. Contact us today to learn more about our products and services!