July 17, 2023

Why Delivering a Successful Chatbot is Harder Than People Think 

We are now all used to being able to find information, get help and possibly process transactions using chatbots. However, even with the current hype around ChatGPT and other advanced generative AI-based chatbots, why does it remain so difficult to deliver even a basic chatbot successfully?

Business Benefits of Chatbots

Chatbots have the potential to deliver tremendous business benefits by being able to interact with users through natural language as if the chatbot were a person, but with the advantage of instantaneous access to more knowledge, information and data than even the best subject matter experts. 

The conversation can be immediate or running in the background, and there will be no queue. A chatbot can support interactions with very many users in parallel, with far fewer human agents in support, and with a superior ability to systematically review outcomes in order to refine content and dialogues in a virtuous circle of improvement. 

Chatbots can even improve how they match questions and answers by learning from the conversations themselves.

The breadth, therefore, of successful uses can range from expert help, support and troubleshooting to customer-facing lead handling and account management, with a superior user experience, better outcomes and lower expense.

Characteristics of a Good Chatbot

Done well, chatbots can provide many benefits adding value to both sides of the interaction: a single point of entry, where ideally the chatbot already knows the user, what they were last doing, and what they are trying to do now. It can respond to requests made in natural language. It can ask follow-up questions, provide contextually relevant FAQs or menu options, show informational text or videos, and ideally be able to resolve the request by launching a transaction.   

If necessary, it should be able to pass the request on to the most appropriate person having triaged the conversation and collected the relevant information, and if it passed the user on to a queue it should do so with an indication to the user of the expected wait time (if any) and with a call back message option.

Chatbot Technology

The technology to enable this level of user experience is becoming wider and easier to put into practice. 

Chatbot Front End

At the front end, there will be a ready-built chat interface for the user which can be integrated with common messaging platforms such as Slack. The chat interface will have built-in functions to make it as simple as possible to create the conversation flow, and it will be able to translate user input into an intent which can then be acted upon. 

There will be other features available in the chat interface which can make the responses more human-like such as remembering the context so that, for example, if the user enters ‘it’ in a follow-up response concerning a contract, the chatbot will know the user means the contract referred to previously. 

Other functions can ignore common typos or recognise the sentiment.

Chatbot Knowledge Base

Behind the chatbot, in order to be able to answer questions, it is likely that there will be a knowledge base which will take a question presented from the front end and return an answer, or possibly alternative answers, based on what is in that knowledge base. 

The knowledge base can consist of documents, web pages, databases, or curated questions and answers, all of which will have been ingested and possibly labelled in some way by users in order to help the knowledge base find the best match to the question passed to it, or it might be a Large Language Model (LLM).

To increase the likelihood of a successful response, chat interfaces, knowledge bases and processing surrounding LLMs (if used) will have other enrichments and functions such as being able to recognise keywords or concepts or search for data important in that interaction, or it might involve prompt engineering in the case of LLMs to heavily guide the responses and scope that the LLM will use to retrieve its answer. Depending on the configuration the answer retrieved may then itself be passed through an LLM to provide the final finessed response.

Chatbot Back-end

The other main element in the solution will be a database holding a user profile and any contract or other relevant information. This can be used to derive responses, perform authentication, work out the next steps and possibly provide some kind of interface to launch transactions.

Chatbot Challenges

While there is a proliferation of ever more effective solutions which can make it easier to spin up an impressive proof of concept relatively quickly, there remain many challenges, pitfalls and difficult trade-offs involved in successfully deploying, expanding and then maintaining a chatbot solution.

Scope and Expectations

The primary area that requires careful consideration up-front surrounds scope and expectations. The objective and outcomes, particularly to start with, must be understood and agreed upon between business and IT, including the initial and ongoing commitment required from the business. 

The challenge here is that to be most effective a chatbot has to be supported across many business functions and that can result in a relatively simple use-case ballooning in scope. 

If possible, try to keep the scope small, with sensible boundaries well-supported by effective hand-offs, and consider an internal use-case initially rather than starting out on external users. 

Also, whilst it sounds obvious, to be successful the chatbot has to be able to provide something superior to what users experienced before, and it must be designed from the point of view of the user, not the department that is looking to ‘offline’ their workload. 

Set KPIs, Measure and Review Effectiveness

As part of agreeing on the scope of a chatbot, it is important to define what success will look like with formal KPIs, and then instigate regular reviews to make sure feedback and dropouts are reviewed once the chatbot is up and running. Be prepared for initial high use to fall back when, for example, in an internal use-case, users bookmark the wonderful resources uncovered by the new chatbot, instead of going back to the chatbot. 

Also, for a chatbot to be successful it is likely that it will have to be supported by constant marketing and incremental enhancements, particularly if it is internal and the use is optional.

Business Commitment

Another common challenge experienced across organisations deploying chatbots is that the commitment required from the business can often be underestimated. Although a chatbot might look at the outset like an IT project, the initial and ongoing commitment from the business can arguably be greater than traditional IT projects. This is because the business must be closely involved in defining the conversation flows in the greatest detail and then must commit to maintaining the dialogues and content going forward.

Keep it up to Date

A chatbot will only be used if it is trusted to have accurate information. Therefore it is essential to make sure that the content is accurate and up to date. If the knowledge base on which the chatbot depends is not accurate, useful, current and consistent then it can quickly fall into disrepute or cause problems.  

Business subject matter experts must curate and label the data in the knowledge base and maintain the content in perpetuity. Initial enthusiasm can dwindle especially if the same content has to be repeatedly reviewed and re-worked. 

Foundation Models

To help with the challenges above, there have been recent enhancements in the chatbot knowledge base space to provide pre-trained foundation models which can, in theory, understand documents the first time without as much labelling and enrichment from users, and to handle updates with less user intervention. 

But even with this enhanced ability, the responsibility will still lie with the business to ensure that no inaccurate or out-of-date documents are being brought in. It is worth keeping front of mind that, unlike a live agent, a chatbot won’t be able to identify if there’s a problem with the content.

If the scale and management of the content is a challenge then consider a content management strategy as a complementary solution that can help keep documents current.

Reliance on Developers

Another potential pitfall can occur when the conversation flows have been set up and have to be subsequently maintained by developers.

If this is the case then as the scope of the chatbot expands then so too does the development team to avoid a potentially difficult backlog of urgent changes. To reduce the reliance on developers, ensure your chatbot solution adequately supports the capability of ‘no code’ user-definable conversational flows and interfaces.  

Whilst this can further increase reliance on the business, in the long run, it will be more efficient and promote agility.

Agile Project Team

It is important to get the project structure and staffing correct. Even the smallest chatbot pilot should be set up within a modern agile management system with a product owner to keep the business and IT in step.  

On the IT side, even a pilot will need a CI/CD pipeline to check for compliance and vulnerabilities. Automatic testing scripts in the pipeline will also be necessary but can look unusual given the textual nature of the input and outputs.   

One tip on the technical design is that it is worth recognising up-front that as the chatbot expands in scope, different departments will want to manage their own content separately, and that they will want to flag content as internal only for their department, internal for the business unit, perhaps internal within the wider business group, and public, and that the automatic test cases must make constantly checking this a priority. 

Remember a chatbot is likely to bring confidential and external content together in one place, but unlike a person, it won’t recognise a problem if confidential information is being exposed. 

Additionally, it will be important to ensure that due consideration is given to GDPR and accessibility.

Transactional Data has to be ‘Live’

If a chatbot needs to look up information or launch transactions, it will need to access account data, in real-time, often combining data sources and providing validation (or calling microservices to do that). Very quickly a simple look-up or transaction can start to involve complex information processing and complexity.

Transactions Require Authentication

In order to run a report, statement, or launch transactions, a chatbot must be able to access the authentication and access profile for the services being requested, adding another layer of complexity.


In order to meet user expectations, even a simple chatbot needs to be scoped carefully, will require considerable commitment from business users, and can present numerous technical challenges. Chatbots using Generative AI and LLMs bring their own challenges discussed here.

However, as the applications supporting chatbots evolve, it is undoubtedly becoming easier to realise the benefits of chatbots which really can provide enhanced value and improved efficiency internally and externally.

How Finativ can help

Finativ has a wider view of how to design and deliver a successful digital transformation which can be found here.

Finativ provides deep industry experience and expertise and can help with your business transformation. We help drive successful outcomes in areas such as operating model design, data strategy, digital architecture and organisational landscape.  

We can support new programme set-up and governance, help rescue failing projects, or provide oversight, troubleshooting and executive sounding-board roles.

Simon Potts Finativ

Simon Potts

Simon Potts brings a wealth of expertise and experience to the field of AI and digital transformation.

With over 20 years of experience at IBM Financing, he has been the global lead on AI and digital transformation. Simon is recognised as an accomplished technology leader, holding global roles covering IT strategy and digital transformation.

Subscribe to INSIGHT

Subscribe to our newsletter to receive each issue directly in your inbox. Unsubscribe at any time. View our privacy policy here.
Subscription Form

Related Posts