[Preview] Understanding HCI Practices and Challenges of Experiment Reporting with Brain Signals: Towards Reproducibility and Reuse
Felix Putze, Susanne Putze, Merle Sagehorn, Christopher Micek, Erin Solovey
CHI'22: ACM Conference on Human Factors in Computing Systems
Session: Brains, Senses, and Virtual Reality
Abstract
In HCI, there has been a push towards open science, but to date, this has not happened consistently for HCI research utilizing brain signals due to unclear guidelines to support reuse and reproduction. To understand existing practices in the field, this paper examines 110 publications, exploring domains, applications, modalities, mental states and processes, and more. This analysis reveals variance in how authors report experiments, which creates challenges to understand, reproduce, and build on that research. It then describes an overarching experiment model that provides a formal structure for reporting HCI research with brain signals, including definitions, terminology, categories, and examples for each aspect. Multiple distinct reporting styles were identified through factor analysis and tied to different types of research. The paper concludes with recommendations and discusses future challenges. This creates actionable items from the abstract model and empirical observations to make HCI research with brain signals more reproducible and reusable.
WEB:: https://programs.sigchi.org/chi/2022/program/content/70506
Presentation Video:: https://www.youtube.com/watch?v=PxzuXJr50qw
DOI:: https://doi.org/10.1145/3490554
Video previews for CHI 2022 Interactivity