On the insights tab of your AI zone, if you scroll down a bit, you will find the quality control report section on the right lower corner. The reports you generate here help you investigate whether the AI has been trained with sufficient and accurate information. If not, you can always add additional sources and update existing ones.
Let’s take a look at the steps you can take.
Add control questions
First, you will have to add some control questions along with your desired answers. You can start with a couple of questions and add more as they come to mind.
A quick way to get started is by taking a look at the Most common questions in the last 30 days block in your AI zone insights. You can review the questions that get asked the most and click the icon with the + sign to add them to the quality control center.
Checking the answer quality for these popular questions is a good way to ensure that you have a solid base to work from.
Tip: You can also your customer support team for suggestions as they are the experts when it comes to FAQs.
Generate a quality control report
Once you’ve added some control questions and answers, you can generate a report via the Reports tab. This is also where you will find a list of your reports over time.
Review the quality control report
You can then click open report to view the generated answers and see how they compare to your control answers. Each answer receives a score which makes it easier to review at a quick glance. This way you can focus on the answers that have a low score and explore why. There will also be an explanation of the score for each answer.
In the screenshot below two interesting examples are highlighted.
The first one is about whether Unless offers a free trial. And while the correct answer is no, the generated answer is yes. This is likely because we used to offer free trials in the past and must have forgotten to remove some mentions to this on the website. In this case, we can click the i icon for information, see the sources that were used to generate this answer, and update it. We can then re-run the report and see if the score improves.
The second highlighted example is one where the generated answer is correct but it happens to contain additional information when compared to the control answer. However, the extra information is still correct so the score remains high at a 7/10.
That’s all you need to know, time to add some questions and generate a report!
