Continuous Automated Testing and Training Framework for Watson Conversation
For cognitive projects, a majority of the effort is spent on Conversation design. It is usually is a continuous process as it needs to be. Continuous training and Continuous test data validation with End-user trials is key to a successful rollout.
So how do you ensure that a small change to your conversation design won’t break everything else? The key is regression testing.
How do you ensure that a continuous change in your re-training efforts is working as expected?
Following are some of the areas that need to be tested/validated on a continuous basis.
Below are some the challenges that need to be addressed to scale a Conversation centric application.
To support best UX Practices for building Chatbots, Human-centric dialog design is key so that the end user finds what it is looking for in a guided process. This means designing and testing multi-entity scenarios.
A conversation (or chat) is a chain of statements exchanged between two or more individuals. Mostly, conversations happen on a particular topic or in a situation. Whatever the topic or situation is, Context is very important to maintain the state of a conversation. Making sure your training process or validation of test data supports context is a challenge.
Nexright Watson Conversation testing/training framework
We have developed a testing and training framework to address some of the above needs. Nexright Watson Conversation testing/training framework support both technical and business users. For example
- User Interface for Business users
- Jenkins (open source continuous integration tool) for Technical developers
The framework supports some standard areas including Response accuracy, Intent identification, Entity Identification, Confidence level. The user interface allows business SME or Domain experts to identify all unmatched intent.
