Human-AI interactions support explainable AI

Because machine learning models are opaque, explainable AI (XAI) should reveal how and why machine learning models work. 

• Source: Figure 2. XAI Concept - New machine-learning systems will have the ability to explain their rationale, characterize their strengths and weaknesses, and convey an understanding of how they will behave in the future. 

With XAI, a human-in-the-loop cultivates symbiosis between human and AI agents which leads to:

What human-computer interface techniques might translate machine learning models into explanations for humans?

I benefited from studying research published by Sarath Sreedharan, Anagha Kulkarni, and Subbarao Kambhampati¹. They assert that AI agents need to go beyond planning and models of the world and consider mental models², a concept from the theory of mind.

The following graphs attempt to describe a portion of human-aware AI interactions. Although the capabilities are not yet realized, the propositions are situated in the present tense.

Please share your feedback and make use of this material in accordance with the Creative Commons license.³

How do human-AI interactions affect XAI?

How do mental models influence agent architecture?

What do AI and human agents understand?

What is the role of explanations?

1. Explainable Human--AI Interaction: A Planning Perspective. Synthesis Lectures on Artificial Intelligence and Machine Learning, January 2022, Vol. 16, No. 1, Pages 1-184

Zahedi, Zahra, Sarath Sreedharan and Subbarao Kambhampati. “A Mental-Model Centric Landscape of Human-AI Symbiosis.” ArXiv abs/2202.09447 (2022): n. Pag.

2. Wikipedia contributors, "Theory of mind," Wikipedia, The Free Encyclopedia, (accessed July 18, 2022) - Theory of mind (Wikipedia definition)

3. Attribution-ShareAlike 4.0 International (CC BY-SA 4.0)