In my previous blog "The Economic Benefits of Information Standardization"I made a comparison between standardization within industries, such as production processes, and the creation of management reports and dashboards. Where standardization will lead to a lower cost of ownership.
In this blog I focus on the other side of the spectrum; the information consumer or the decision maker. The first part is a summary of scientific articles that have been published about the information consumer's ability to retrieve relevant information in an efficient and effective way. Or in plain language; will less time be needed to perceive the information and with fewer errors?
Researchers from the following universities: University of Applied Science Upper Austria, Saïd Business School - University of Oxford and Modul University Vienna have done research in recent years that has resulted in several articles (see bibliography and authors at the end of the blog).
Based on previous findings, the researchers used eye tracking technology to conduct their research. Therefore, they observed both professionals and students studying business administration in a laboratory setup.
In addition to the error rates and task execution times, eye tracking focuses on how the observers read the visualizations and make the presented stimulus meaningful, a prepared visualization in the form of a management report.
One of the things measured with eye tracking is the fixation of the eye on the focus areas (AIO's); how long and also how often an observer looks at an AIO. Other measures are: saccades (shifts between fixation location, during the movement of the eye no information acquisition can take place), pupil (pupil diameter increased in states of high mental strain) and blinking (blinking is said to indicate information processing).
Let's make a simple setup regarding management reports in 2 different layouts. In the example below a heatmap is shown with the fixations and AIO's of one layout.
Where the heatmap shows the concentration, the fixation path shows how the eyes move from one AIO to the other. For example visualizations using legends instead of labeling in the graph itself will lead to an increased eye movement between both AIO's (the graph and the legend).
And as a result, data on effectiveness (task accuracy) and efficiency (task duration) can be collected, measured and tested for significance across the entire sample of observers.
In the above sample, a clear increase in accuracy and decrease in time was measured when a layout was selected with improved visualization and in accordance with the task to be performed.
Efficient and effective information processing and decision making depends not only on the chosen visualization. Other factors also influence the cognitive load and result in efficiency and effectiveness.
It goes without saying that the complexity of the task to be performed (looking up a single value or making a comparison) and the complexity of the data influences the cognitive load, while the knowledge and experience of the information consumer makes it possible to cope with a higher cognitive load.
In the situation that the cognitive load becomes too high, an information overload occurs. The load is the sum of information and sound. This makes the efficient and effective processing of information more difficult. It takes more time and is more prone to errors.
Let's distance ourselves from the research and project it on self-service BI. In this case, both relevant and irrelevant (noise) information is offered to a group of information consumers. The level of task and complexity required per individual information consumer is not taken into account, as the created dashboards are generic - make one-time, by many. This means that self-service BI leads to a higher cognitive load, or even an information overload. It will be less efficient and effective than a (management) report created to support a specific task for a specific information consumer.
The publications cover various aspects and many hypotheses are tested. Some were rejected, while others were tested positive. There are too many findings to share, among just a few:
- Layout optimization based on human cognition, reduces reaction time and reduces the amount of (re)fixations required.
- Standardisation and repetitive reporting help to quickly draw the right conclusion. Where standardisation should not be an obstacle to changes in the direction of a perception-optimised layout.
- Visualisations aimed at the public will in turn increase the capacity of information perception and improve decision-making.
- Cognitive strain is very important for practice. System and interface designers must prevent an overload of information for the users concerned.
The Link with IBCS
Although the studies did not use IBCS-specific reports as stimuli, there is a parallel in the findings and the concept of IBCS.
IBCS is based on 3 pillars: conceptual, perceptual and semantic rules. For the perceptual part, it is clear that good visualisation will help to reduce the cognitive load and will lead to more efficient and effective information processing and decision making. Perceptual rules are increasingly accepted in the design of dashboards and reports.
The conceptual rules - SAY and STRUCTURE - will help in the sense that the focus is on the message, i.e. the task to be performed, and by giving a good structure to the content that what is complex is easier to digest (data type).
In addition, the semantic rules are an important factor of influence. On the basis of these rules, a standardisation is introduced within the (management) reporting that frees up the capacity of the recipient's working memory. The capacity of the working memory is limited. Through repetition and standardization, we are able to make use of our capabilities of long-term memory, which is "unlimited" in capacity. This is what you see in engineers and architects, for example. Based on their standardisation in the industry, they are able to read blueprints at a high condensation level without information overload.
Many thanks to the University of Applied Sciences Upper Austria, Austria, for sharing the publications and in particular to Ms Lisa Falschlunger.
This blog is based on a personal interpretation of scientific publications. I have tried to be as precise as possible.
University of Applied Sciences Upper Austria, Austria
Said Business School, University of Oxford, UK
Modul University Vienna, Austria
Report Optimization using Visual Search Strategies An Experimental Study with Eye Tracking Technology (2014)
Visual Representation of Information as an Antecedent of Perceptive Efficiency: The Effect of Experience (2016)
Cognitive Differences and Their Impact on Information Perception: An Empirical Study Combining Survey and Eye Tracking Data (2015)
InfoVis: The Impact of Information Overload on Decision Making Outcome in High Complexity Settings (2016)
Using Gaze Behavior to Measure Cognitive Load (2018)