Research

For object detection models, I study how to visualize which regions in an image influenced each detection result. Object detection estimates not only the categories of objects in an image, but also their locations. As a result, directly applying explanation methods designed for classification models can produce explanations that respond strongly to the background, surrounding regions, or even another object, rather than to the detected object itself.

To address this problem, my research emphasizes generating explanations that correspond to each detected target, rather than explanations for the entire image. Specifically, I estimate important regions so that they reflect not only what the model detected, but also which object at which location served as the basis for the detection. This makes it possible to separately visualize the decision evidence corresponding to each detection result, even when multiple objects exist in the same image.

These explanations make it easier to check whether the model is making correct judgments based on the shape and appearance of the target object, or whether it is relying on the background or surrounding context. In particular, when false detections or missed detections occur, I aim to help people analyze their causes and connect that analysis to model improvement and reliability evaluation in real-world environments.[ICIP 2022][IEEE Access 2025][CVPRW 2024][CVIU 2026]

The decisions of image-recognition models are not always determined by a single visually salient region in an image. Parts of the target object, the background, surrounding objects, colors, shapes, and spatial arrangements can become important cues only when multiple regions are considered together. For example, even if one region has only a small influence on the decision by itself, it may greatly change the model's prediction when it appears together with another region. Conventional simple importance maps have difficulty capturing these interactions between regions.

To address this problem, my research uses ideas from game theory to analyze how each region in an image contributes to the model's decision. In particular, I evaluate the contribution of each region based on Shapley values, and further incorporate the concept of Interaction to capture not only the importance of individual regions, but also the effects produced by combinations of multiple regions. This makes it possible to explain not only which parts the model is looking at, but also which combinations of parts it uses as evidence.

At the same time, applying Shapley values and Interaction exactly to an entire image creates an enormous number of region combinations to consider, resulting in very high computational cost. My research therefore aims to realize accurate game-theoretic explanations with practical computation by efficiently searching for combinations of regions that are strongly related to the model's decision. This enables more faithful visualization of complex decision evidence that conventional heatmaps can easily miss.[CVPR Highlight 2026]

Visual explanations such as heatmaps can intuitively show which regions in an image the model focused on. At the same time, people still need to interpret what about those regions influenced the decision. For example, even if the same bird region is highlighted, a heatmap alone does not clearly reveal whether the model is using the color of the feathers, the shape of the beak, or the waterside background as evidence.

To address this problem, my research works on explaining the visual features that influenced the model's decision in natural language. Rather than simply describing the content of the image, I aim to express the features that supported the model's prediction as words, making it easier for people to understand more concretely why the image was classified into a particular class. This can make explicit decision evidence such as color, shape, object parts, background, and context, which may be difficult to understand from visual explanations alone.

What I particularly emphasize is not generating plausible explanation sentences, but obtaining explanations that are actually connected to the model's output. Natural language explanations are easy for people to understand, but they also risk becoming general image descriptions or explanations that are not directly related to the model's decision. I therefore evaluate how well the explanation text represents concepts that influence the model's prediction, with the goal of realizing natural language explanations that are faithful to the model's decision.[CVPR Findings 2026][arXiv 2026]