As Artificial Intelligence (AI) grows in power and availability, it is becoming an essential tool for businesses. As well as its ability to automate repetitive and manual tasks, it also has the potential to offer intelligent insights to improve decision-making.
For organisations to remain accountable and in control of these advancements, it is important for AI to be as transparent as possible. However, at present, not enough is being done to address the lack of understandability and explainability of AI, which is unsustainable in a business context.
Lack of transparency in AI models is referred to as the ‘black box’ problem. Typically, the inner processes of machine learning (ML) models are obscured since only algorithmic inputs and outputs are visible. In particular, this is true of complex structures such as Deep Learning (DL) neural networks, which consist of many neural nets of multiple neurons. This is also the case with algorithms simultaneously optimising many variables to map geometric patterns.
As a consequence, it is difficult to discern how variables are being combined to reach conclusions. This results in the inability of the users of AI outputs and even model designers to understand or explain AI insights.
This lack of transparency is problematic due to many reasons, including:
All of the above can lead to grave organisational costs, which might not only be purely monetary. There could be performance issues leading to operational inefficiencies and business disruptions, or even reputational damage and legal repercussions.
It is clear that there is work to be done to combat opacity in AI systems within organisations. An increasing field of research into ‘white box AI’ is emerging, which promotes a system that is understandable, justifiable and auditable by humans. This should offer insight into how models behave and which factors affect their decision-making.
From a technical standpoint, there are interpretability techniques which can create clearer observability of the inner workings of an AI system. One such way is to enforce model constraints, including limiting development to decomposable or linear models. Additionally, it is possible to adjust inputs to infer how they will affect model behaviour or to use additional explanation neural networks to probe.
However, it is insufficient for solutions to be understandable by technical experts alone. Since AI is used to carry out specific business functions within organisations, it is also necessary to remove barriers to visibility and accountability for wider stakeholders. As well as prioritising stakeholder education, this requires a culture of transparent AI, defined by the human actors involved.
Transparent AI requires the active involvement of both technical and non-technical human stakeholders in the design and process of an AI journey. It is necessary to have a Human-In-The-Loop (HITL) framework in place, which incorporates meaningful human expertise into the system. Directly involving human stakeholders into the workflow provides stakeholders with increased insights into model functionality as well as introducing clear structures for accountability.
In Nexus FrontierTech, we design our AI solutions to include a domain expert in the process, which has come to be known as the Expert-In-The-Loop (XITL) framework amongst our clients and stakeholders. With this, we incorporate expert feedback loops and use these to inform model improvements (read more about collaborative intelligence here).
A transparent XITL architecture leads to sustainable and scalable AI initiatives. Traditionally, it can be difficult to upgrade these technologies in line with changing demands. However, when humans are an inherent part of the architecture, the technology will grow organically in line with requirements. Moreover, transparent models are more transferable, since it is easier to reuse and fine-tune them for different contexts across the business. As a result, companies are likely to realise greater returns from their AI initiatives.
To make this possible, it is necessary to have the correct tools available including User Interfaces (UIs) such as dashboards so that humans are provided with a visual way to understand and validate outputs. Companies can consider using an end-to-end AI platform to optimise efficient integration of human stakeholders into their initiatives.
Lack of transparency in AI systems can have serious ethical ramifications, impact the scalability of solutions, and present a barrier to organisational adoption of such technology. It is important to promote understandability and explainability of how AI models reach conclusions, especially in cases where important tasks and decisions are being handled. This is both a technical and a business consideration; as well as improving the interpretability of AI for developers, it is equally important for companies to have a coherent framework in place to allow visibility and feedback from all relevant stakeholders. With this focus on implementing transparent AI solutions, companies will succeed in retaining control of AI and developing sustainability for the business.
1 St Katharine’s Way,
E1W 1UN, London, United Kingdom
6 Battery Rd, #03-68,
The Work Project @ Six Battery Road
Studio 1006, Dreamplex Thai Ha, 10th floor 174 Thai Ha Str, Dong Da Dist
Hanoi 100000, Vietnam