The power of visual influence
What motivates or impels the human eye to fixate on a target and how, then, is this visual image perceived? What is the lag between our visual acuity and our reaction to observation? In the burgeoning field of immersive virtual reality (VR) and augmented reality (AR), connecting these dots, in real time, between eye movement, visual targets and decision making is the power driving force behind a new computing model developed by a team of computer scientists at New York University, Princeton University and NVIDIA.
The new approach determines a user’s real-time reaction to an image or scene based on their eye movements, specifically saccades, the lightning-fast eye movements that oscillate between dots before settling on an image or an object. Saccades allow frequent changes of attention to better understand one’s environment and locate objects of interest. Understanding the mechanism and behavior of saccades is key to understanding human performance in visual environments, which is an exciting area of research in computer graphics.
The researchers will present their new work, “Image Features Influence Reaction Time: A Learned Probabilistic Perceptual Model for Saccade Latency,” at SIGGRAPH 2022 taking place August 8-11 in Vancouver, British Columbia, Canada. The annual conference, which will be in-person and virtual this year, spotlights the world’s top professionals, scholars, and creative minds at the forefront of computer graphics and interactive techniques.
“Extensive research has recently been conducted to measure visual qualities perceived by humans, especially for VR/AR displays,” says the paper’s lead author, Qi Sun, PhD, assistant professor of computer science and engineering. engineering at New York University’s Tandon School of Engineering.
“But we have not yet explored how the content displayed can influence our behaviors, even significantly, and how we could possibly use these displays to push the limits of our performance that are not possible otherwise.”
Inspired by how the human brain transmits data and makes decisions, the researchers implement a neurologically-inspired probabilistic model that mimics the accumulation of “cognitive trust” that leads to human decision and action. They conducted a psychophysical experiment with parameterized stimuli to observe and measure the correlation between image features, and the time it takes to process them to trigger a saccade, and if/how the correlation differs from that of visual acuity.
They validate the model, using data from more than 10,000 user experience trials using an eye-tracking VR display, to understand and formulate the correlation between visual content and the “speed” of decision-based decision-making. on the reaction to the image. The results show that the new model prediction accurately represents human behavior in the real world.
The proposed model can be used as a metric to predict and modify users’ eye image response time in interactive computer graphics applications, and can also help improve the design of VR experiences and player performance in esports. . In other industries such as healthcare and automotive, the new model could help estimate a doctor’s or driver’s ability to respond quickly to emergencies. In esports, it can be applied to measure the fairness of competition between players or to better understand how to maximize performance where reaction times come down to milliseconds.
In future work, the team plans to explore the potential for cross-modal effects such as visual and audio cues that jointly affect our cognition in scenarios such as driving. They are also interested in expanding the work to better understand and represent the precision of human actions influenced by visual content.
The authors of the article, Budmonde Duinkharjav (NYU); Praneeth Chakravarthula (Princeton); Rachel Brown (NVIDIA); Anjul Patney (NVIDIA); and Qi Sun (NYU), are set to demonstrate their new method on August 11 at SIGGRAPH as part of the program, Roundtable Session: Perception. The paper is here.
About ACM SIGGRAPH
ACM SIGGRAPH is an international community of researchers, artists, developers, filmmakers, scientists and professionals with a common interest in computer graphics and interactive techniques. A special interest group of the Association for Computing Machinery (ACM), the world’s first and largest computing society, our mission is to nurture, advocate and connect like-minded researchers and practitioners to catalyze innovation in computer graphics and interactive techniques.
Disclaimer: AAAS and EurekAlert! are not responsible for the accuracy of press releases posted on EurekAlert! by contributing institutions or for the use of any information through the EurekAlert system.