- Business Model of chatbot implementation
- Functionality – what chatbot should do
- Chatbot implementation project
- Effects of chatbot implementation
- Further development of chatbot
The chatbot project for the Customer’s brand was initiated in April 2010. It was assumed that in the first place the system would serve the landline clients.
Based on the number of inquiries to the call center and website traffic, it was initially estimated that the chatbot would serve 17,000 clients per month, answering an average of five questions per conversation. It was assumed that the database of specialized knowledge (concerning the Customer’s subject matter) will include about 2000 facts and will gradually increase by about 15% per year.
Analyzing the solutions available on the market, different approaches were considered for the scope of functionality (what functional modules should be included in the solution), maintenance (installation on the Customer’s own or Provider’s servers) and licensing (purchase of a license together with implementation and maintenance or SaaS model – Software as a Service).
Business Model
As a result of discussions with various Providers and negotiations with the selected Provider, it was decided to use the SaaS model. The entire solution is hosted and maintained within the Provider’s infrastructure, and the settlement model is based on the number of effective responses given by the system. This solution, although cost-effective at the stage of purchase, was associated with the risk of lack of information regarding the real number of conversations that the chatbot can carry out. Until now, no one in the telecommunications industry has decided to implement a solution on such a scale. The risk was borne by the Customer – the traffic generated by the chatbot could be very large, which could exceed the budget. The risk was also borne by the Provider, because if the traffic turned out to be too small, the fees received would not cover the costs of the project. In order to secure the interests of both parties:
- upper budget thresholds were established (if the number of calls exceeded, the risk was borne by the Provider),
- minimum thresholds were established (a fixed monthly minimum fee was introduced),
- It was decided to gradually introduce the chatbot, making it available from more and more subpages and at the same time observing the increase in traffic generated by the system.
The adopted model proved to be successful both in 2011 (the year of launching the system) and 2012 when the scope of the project was significantly extended.
Functionality
The theoretical functional capabilities of chatbot solutions are very wide. Systems of this class can not only carry out independent conversation in natural language and answer questions asked by users, but they can also solve more complicated problems of users. Chatbots can also be integrated with IT systems on the Customer’s side, so they can provide information stored in users’ accounts or record data directly in databases. It should be noted, however, that each new functionality complicates the implementation process and lengthens its duration. Before it was decided to implement the whole project, the Customer’s team asked the Provider to prepare a pilot implementation of the chatbot into which a small piece of factual knowledge was introduced. Based on the tests conducted, conclusions were drawn as to the scope of the knowledge base to be introduced into the chatbot. At the same time, it was decided that in the first stage of implementation, the system would have basic functionality (without integration with semantic search engine, live chat, and other IT systems on the side of the Customer). It was also decided not to record an individual image (avatar), choosing one of the standard ones offered by the Provider.
Project implementation
The project was carried out by one of the Customer’s companies responsible for maintaining communication channels with clients and after-sales support. After completing the pilot implementation and testing its capabilities, the work on the content-related knowledge base began. During the first stage, about 300 issues were introduced to the system. Due to the extensive issues that are the answers to some questions, it was decided that in part of the answers the system redirects the questioner to the appropriate sub-page of the website, where there is a description of the issue sought. In the case of some facts, it was necessary to redirect the user to the hotline (due to lack of integration with other IT systems). In the future, this problem is to be solved by integration with live chat and direct connection between chatbot and Call Center consultants.
Entering facts into the factual knowledge base is done by the Customer’s employees using the Administration Panel supplied by the Provider. Next, the facts are “tuned” by the Customer’s employees – the point is to avoid conflicts in the knowledge base and optimal adjustment of the natural language processing engine to the new issues. After introducing the knowledge base, the solution was tested – first, the tests were carried out by the Provider’s employees, and second – by the Customer’s employees.
The system was launched in September 2011 within the “Customer Service” section of the Customer’s website. After launching the solution at the Customer, a team was formed to monitor the functioning of the solution and the introduction of the missing facts. The team monitors, on an ongoing basis, the questions asked by users and the answers given. In case of a wrongly given answer, this fact is immediately reported to the Provider, and if there is no answer in the knowledge base, such a fact is worked out and entered into the system. In the early stage of the solution’s functioning, all conversations were monitored, but due to a significant increase in the number of conversations, at the moment the system logs are verified every “n-th” day.
In 2012, it was decided to expand the implementation of new content areas. Knowledge bases on mobile telephony and knowledge for business clients were created and launched. At the moment, the knowledge base contains about 1500 content-related issues.
During the process of rebranding of the Customer, a recording of an individual avatar image was introduced. The voice of the synthesizer was also changed (specially prepared for the needs of the Customer).
Effects
During the first few months, the number of conversations conducted by the chatbot steadily increased from about 20,000 to over 100,000 per month. Both numbers exceeded initial estimates. It should be noted, however, that the number of substantive questions per conversation is lower than initially estimated (3 instead of 5), while the entire conversation on average includes 12 user questions.
Thanks to the continuous monitoring of the solution, it was possible to increase the quality of answers from 67% of correct answers at the launch of the solution to over 89% after a period of about 6 months.
However, the most important point concerns the effect of reducing the number of calls coming into the hotline. A significant correlation was observed between an increase in the number of calls handled by the chatbot and a decrease in the number of calls entering the hotline. The scale of migration is from a few to a dozen or so percent. The Customer estimates that the cost of a conversation conducted by a chatbot is five times lower than for a similar conversation conducted by a hotline agent.
Development
The project implemented by the Customer in Poland has received a lot of attention throughout the Group. The Group is considering the implementation of similar solutions in other countries around the world. At the same time the works of the substantive team are still carried out. The team, together with the Provider’s team, is getting ready to test solutions including a chatbot accessible from mobile devices and equipping the chatbot with a speech recognition system. It is also being considered to integrate the chatbot with other IT systems of the Customer (access to the client’s contact data, possibility of reporting service cases, etc.) and with the live chat service (possibility of automatic redirection of the conversation from the chatbot to the agent in the Call Center).
Summary
The chatbot implementation project in Poland carried out by the Customer and Provider teams is one of the most advanced and successful implementations of such solutions in the world.
Thanks to the continuous commitment of both teams to improve the quality of the solution and the optimal business model that satisfied both sides of the project, very positive results were achieved both in terms of content (quality of correctly answered questions at the level of 90%) and economics (reduction of the total cost of customer service by redirecting some clients from the hotline to the automated service Provided by the chatbot). At the same time, the Customer Group has become a pioneer in implementing these types of solutions in the telecommunications sector.
