Think like an AI ethics officer: discovering human-AI complementarities

If we look at AI as a tool to be used for tasks, then the overarching goal of striving towards ethically responsible AI breaks down into three more concrete questions:

The first question can be recast through the lens of looking at where to find positive human-AI-complementarity: in what tasks can AI compensate for the weaknesses of human information processing and decision-making and conversely, what tasks can humans compensate for the limitations of AI tools? This question is easy to formulate, but much more difficult to answer, because there is no neat dividing line: in some tasks AI is better than humans, but it may be worse at a task that seems reasonably similar (this is the ‘jagged frontier’ as Ethan Mollick and co-authors call it in their paper Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality). 

In fact, a recent meta-analysis of 370 tasks that compared AI and humans (When combinations of humans and AI are useful: A systematic review and meta-analysis published in Nature Human Behaviour), complicates matters even more. On one hand, there was substantial evidence for human augmentation: in 85 per cent of the tasks, the combination of human and AI performed better than humans alone, suggesting that many tasks would have positive complementarity. 

On the other hand, if you compare the human-AI combination with how good the performance could have been if you had used either AI by itself or humans by themselves, then in 58 per cent of the cases the combination led to worse performance, indicating a negative complementarity. 

And again, the picture for which task you would have positive or negative complementarity was far from conclusive. So if it’s virtually impossible to know in advance for which tasks it would or would not be beneficial to implement AI, what should organisations do?

“There’s only one way to find out.” Go ahead and just do it. Experiment, test, try, see what works and iterate on that. Not in a technology-centric ‘move fast and break things’ approach, but in a ‘fail fast and learn’ approach that places humans and their work at the centre. 

To make this a bit more concrete, let’s look at the following use case from the Sociale Verzekeringsbank (SVB): an organisation like the SVB has a large knowledge base consisting of (among other things) legal documents, procedural instructions, execution manuals and implementation documents that translate the legal framework of social security provision into organisational procedures that describe the procedures related to actions that need to be undertaken in relation to specific client cases. 

If an SVB employee has a specific question for their work, the necessary information is often spread out over multiple knowledge sources, making access and correct knowledge retrieval complicated. AI could potentially be a very useful tool to facilitate easier access to this knowledge base, for instance via chat or voice interaction. This sounds simple enough: build an AI solution using RAG techniques that gives the answer a user is looking for, perhaps allowing for follow-up questions if the initial answer wasn’t sufficient for the employee. 

However, this application becomes a bit more nuanced when looking at it from the perspective of positive human-AI complementarity: if the tool simply provides answers, there could be a risk of automation bias and de-skilling, where the employee accepts the answer uncritically. Eventually they loses the skill to search and process information independently and become overly dependent on the system (for instance analogous to how navigation systems negatively affect human's spatial memory). From that angle, it might actually be desirable to respond to queries by only directing the user to the places where relevant knowledge can be found, perhaps with some instructions on how to combine knowledge from multiple sources, but still forcing the user to read and process the information by themselves. This obviously introduces more friction into the process, but it also keeps human users more ‘on their intellectual toes’ and encouraging them to keep thinking for themselves.

This brings into play questions 2 and 3 from earlier: where can an AI tool improve the work of a human? This can either take the approach of taking the work as the starting point (illustrated by the SVB example), when an AI tool is developed to assist with a specific task (Q2) or when humans play around with an existing AI tool and discover a useful application for it in their work (Q3). 

The approach in Q2 is classical software development, taking user demand as the starting point. The approach in Q3 is more controversial because it has a technology push – ‘if you build it, they will come’ approach to it that is often a guarantee for failure. However, this may be more appropriate for AI than it initially seems, because particularly Generative AI is a general purpose technology like the steam engine, electricity, the PC or the internet. With general purpose technologies, the applications are potentially so broad that the only way to find out what they can be used for is to actually try them, because their inventors don’t know either. In this regard, it is very illustrative that GenAI providers tell their users only what the installation or log-in instructions are, give a few basic examples and that’s it. None have provided a real user manual for how to use them effectively and responsibly. So virtually all guidance – on prompt engineering for example – has come from external users and researchers, not from the technology providers themselves.

That’s why it is crucial to take a ‘fail fast and learn’ approach. Take a clearly focused task, conduct careful limited-scale pilot tests comparing humans, AI, and human-AI-combinations (use a sandbox approach providing the necessary privacy safeguards or ‘shadow AI’-approach to limit operational risk where necessary), monitor the process and performance in detail and evaluate rapidly. Pull the plug quickly on what doesn’t work, learn why it doesn’t work and incorporate those lessons into the next pilot test. This also allows testing for specific requirements, such as how positive complementarity between man and machine can be translated into practical use cases, and experimenting to see how these requirements fare in reality and iterate on that. Only then will developers and users discover where the real human-AI-complementarities are and will we be able to use AI for human benefit.