Speakers

Dusanka is Test Lead and Department Manager in Levi9 IT Services company. In the last nine years, she has been actively involved in several different projects, using various technologies and tools daily. She likes to share knowledge and support various testing initiatives. She is also dedicated to her academic career as a Ph.D. in Technical Science, so she often writes papers for conferences, but also technical articles. She points out current trends and the importance of testing in software development.

From Bugs to Brilliance: How to Test Your Chatbot Effectively

During the previous period, we saw the influence of many AI tools on building and testing applications. However, we also witness that we, as QAs, need to know how to test AI applications. It is not the usual way to conduct traditional testing, where we have one positive and some negative test cases. Testing Generative AI (GenAI) applications is unpredictable and requires a different approach. Can we handle it and say it is tested and of good quality as professionals?
This presentation provides clarification and a path on how to test a chatbot application and guarantee its quality, clearly showing how to handle risks. The plan is to explain through the example of a Retrieval-Augmented Generation (RAG) application how it is implemented, how to use a vector database, and how to compare retrieved and generated content to achieve high-quality testing results.
We will delve into the use of agents and tools that facilitate the testing process, breaking down the application’s documents into manageable chunks for thorough examination. The presentation will cover the importance of vectors and similarity search in ensuring the accuracy and relevance of the generated content. By leveraging these advanced techniques, QAs can effectively test AI applications, ensuring they meet the highest standards of quality and reliability.
Additionally, we will highlight the differences between testing GenAI and traditional applications. Unlike traditional testing, which relies on predefined positive and negative test cases, GenAI testing involves handling unpredictable outputs and ensuring logical consistency, factual accuracy, and contextual relevance. This requires innovative testing strategies and tools to manage the unique challenges posed by AI applications.