There have been many discussions about whether full automation or the human-in-the-loop (HITL) approach should be applied to realize AI ideas.

Regarding the development of an AI model, the interaction between humans and machines is not limited to working together on training data sets. Currently, no AI model can reach 100% accuracy. However, the most effective systems are ones that facilitate a continuous collaboration between the two parties, with machines learning from human decisions and feedback to become smarter. HITL is an approach to AI that enhances the collaboration between humans and machines within a model, allowing people to play an important role in reinforcing an AI solution’s performance.

The ultimate goal of an HITL AI solution is to outperform both a human and a machine when working independently from one another. The machine processes a huge amount of data which a human cannot handle within a short period of time. But currently, there has been no AI model that can reach 100% accuracy rate. With HITL, when the machine fails to perform a task accurately, a human intervenes to give annotations, allowing the algorithm to learn from them and give better results by time.

Human-in-the-loop (HITL), basically you can say, is the process of leveraging the power of the machine and human intelligence to create machine learning-based AI models.

One of the most successful applications of HITL is in the healthcare industry. In an objective evaluation of cancer diagnoses by AI and pathologists conducted by Beth Israel Deaconess Medical Center (BIDMC) and Harvard Medical School (HMS), the accuracy rate of the automated diagnostic method was approximately 92 percent, while that of a human pathologist was at 96 percent.

“When we combined the pathologist’s analysis with our automated computational diagnostic method, the result improved to 99.5 percent accuracy,” said Andrew Beck, Director of Bioinformatics at the Cancer Research Institute at BIDMC and an Associate Professor at HMS. “Combining these two methods yielded a major reduction in errors.”

This approach can be applied to many other sectors and change the way both machines and people work. When weighing full automation and HITL for your business, take into consideration the practical benefits of HITL that have made it more and more popular among companies developing and evaluating AI ideas.

Some of the benefits are stated below.

Increased transparency

Despite performing accurately, many AI applications today are black boxes that lack the ability to “explain” the reasoning behind their decisions. This is risky, especially in evidence-based or highly regulated industries, such as finance and banking. For companies that need to establish a governance framework to avoid risks, the HITL approach can be very effective. Businesses will be able to see how AI systems arrive at a given outcome, taking these decisions out of the black boxes. The process will no longer be a hidden spot for humans.

Empowered systems

In the beginning of HITL application, a continuous feedback loop is required for humans to validate the model’s output and for the model to learn then improve itself from that. And at some point, with enough tuning, machine algorithms can maximise the accuracy without constant feedback from humans. In this way, any machine learning model and AI solution can benefit from human intelligence fed into the feedback loop.

In order to achieve this, many questions need to be answered first while designing an HITL AI model. What outputs need to be validated by humans before being integrated into the existing workflows? Which positions should be involved in the loop and at what level? When do humans need to be notified for intervention? How should the user interface be designed so that humans can seamlessly give feedback?

The specific HITL AI models can vary by case, but there is a principle to keep in mind: break down the tasks and find the junctions to incorporate human interaction.

Better handling of lack of data

Conventional machine learning models need a large number of labeled data points to produce accurate results. Even better algorithms and technological advancements in recent years cannot entirely replace the role of data in AI training. Vast datasets are required for a machine to understand and be able to perform a task.

The fundamental problem here is that not so many companies are aware of a need for training datasets in advance, and preparing for them only after picking an AI solution is usually time-consuming and inefficient. Some may seek public datasets, but more often than not they are too general for a specific business case.

With the HITL approach, an AI solution can be developed, trained and used sooner. Less time and human resources are spent on creating training data. The focus will be more on monitoring the system and giving it feedback for optimisation.

Despite all the aforementioned advantages of HITL, you should also consider the downsides of it:

• Humans can make poor choices due to bias or emotional clouding
• Humans can slow down a process by considering for too long to take an action
• Humans can make errors in the actions they take

Don’t know which approach to take? You can get a guide to developing and evaluating AI ideas in our upcoming workshop AI-Enabled Business: A Proven Framework To Drive Scalability And Reusability With AI.