Watson Conversation testing framework
We live in an age of automation, where we are continuously interacting with systems that are trained to interact with human beings in a lot many different ways. Be it an e-commerce website or a customer care service, be it a marketing platform or a social networking site, we can find these automation services in almost every sector these days.
One of the most common and trending automation technology that’s being integrated into several platforms is a cognitive Chatbot service, that can interact with users in a human manner and help them to get what they are looking for. These systems allow the host to interact with customers or clients and help them to resolve several problems, without even manually interacting with them. The problem could be as simple as an item look up or could be a bit more evolved to book a flight, this service can help attain that need.
Watson Conversation Service
When it comes to cognitive technology and service, IBM provide a whole bunch of services bundled within IBM Watson system. IBM Watson is a cognitive system that enables a partnership between people and computers. Watson cognitive technologies can think like a human and process the human input and answer or consume them appropriately.
One such service is the IBM Watson™ Conversation service. It provides a Chatbot functionality which is interactive and is powered by artificial intelligence and is designed to simulate human conversation. We can integrate this service into an application that understands natural-language input and uses machine learning to respond to customers in a way that simulates a conversation between humans. The host need to model and implement their design into the Conversation service platform for it to understand the appropriate human input and respond to them accordingly.
Cognitive Challenge
Though a properly modeled and designed Conversation service looks flawless and overwhelming when comes to life when used in an application, but to get to a state like that the model needs to reviewed and tested countless times to make it stable enough to handle different situations and use cases.
Even though some components could be modeled by simply uploading a pre structured CSV document, majority of this process is a manual process. This process is time-consuming, error prone and cumbersome, which can make the designing and development process a bit frustrating if lot of information need to be handled.
For cognitive projects, the majority of the efforts is around data quality, data ingestion and testing the overall model with multitude of user inputs. Though the process of maintaining data quality and data ingestion can’t be completely automated, testing of the model certainly can be.
Testing the Conversation service with a batch of input is something that has consumed a lot of our time and pushed us to come up with a solution to automate this activity. We at Nexright has come up with such an application that can take a bunch of user inputs all at one time in a pre structured CSV and test them out individually by making API calls to Conversation service.
Nexright Conversation service testing framework
This test app is a UI interface that can be used to test out the service model. Be it an architect, a developer or the business user or anyone else for that matter, this app can be used by anyone who intend to test the service. Below parameters can be tested out using this app.
- Response accuracy
- Intent identification
- Entity identification
- Confidence level
Conversation Test score
The test score will tell you how efficient your model is and where we have scope for improvement. Multitude of requests with different variations can be submitted to the app and the app will review the requests and their corresponding responses to show how efficiently the model is behaving.
The test result will show you whether the model is good enough to be deployed into real life activity or not.
We can even test out each and every individual request and its corresponding response from Watson if the need be. This app can help to perform model variation as well as pattern variation and overall can help to design the chatbot a lot better and stable.
